ideolalia

shaders in clojure, revisited

A number of months back, I wrote about how to use OpenGL shaders in Clojure, using Penumbra. This introduced an s-expression representation of GLSL, which allowed for an idiomatic specification of shader code. It didn’t, however, cut down on the boilerplate that surrounds the OpenGL programmable pipeline.

There are two steps in the programmable pipeline: the vertex shader and the fragment shader. The vertex shader transforms geometry from the world-space to screen-space. The geometry is then discretized into pixels, and then each pixel is processed through the fragment shader. For a more in-depth look at this process, check out this set of articles.

This is referred to as a pipeline because data flows from the vertices to the vertex shader to the fragment shader. Since these shaders are meant to be swappable, the vertex shader has to declare what data it passes the fragment shader, and the fragment shader has to declare what data it receives from the vertex shader. Mismatched declarations are allowed, which can be useful, but mostly it’s just redundant and increases the potential for errors. If we’re willing to treat the pipeline as a single entity, our specification can become shorter, simpler, and much more straightforward to develop.

So, let’s revisit the marble teapot:

In the previous example, we used the Perlin noise lookup that’s built into GLSL. This approach had a lot of limitations; this built-in function only works properly on a fraction of all hardware, and it can only be used within the vertex shader. As a result, the marble coloration was per-vertex rather than per-pixel, and looked a lot less detailed.

To create our own per-pixel Perlin noise implementation, we need a block of 3-D random noise. We can get this by creating a 3-D texture, which is seeded with random numbers from -1 to 1. To perform a per-pxiel lookup, each pixel needs to know its location in the world space. This is different than its location with respect to the camera; if we used the location in camera-space, our teapot looks like this:

We will also need to know the normal on a per-pixel basis, to allow for per-pixel lighting. These two values, along with the transformed screen-space location, are represented as a hash-map at the end of the vertex shader. These values are effectively the “return value” of the vertex shader.

Notice that in the fragment code, we reference both position and normal, which are defined in the vertex shader. The % variable represents the 3-D texture that’s seeded with random values. Also notice that while this looks a lot like Clojure code, it uses operators like += and *= that belie the underlying C implementation. The above code expands to this, and the complete code can be found here.

Stay tuned for future announcements. I hope to have vertex buffers implemented in the next week or two.

søren

   

creating a simple game in clojure

There’s a certain type of person who, upon seeing a new language, feels compelled to write a game with it. Functional languages represent a unique challenge in this respect, as has been written about at some length elsewhere. Clojure adds an interesting wrinkle to this problem, in the form of its Java interop. If a game consists of a thin layer of Clojure wrapped around a full-featured Java game engine, is it actually “written in Clojure”?

Without getting bogged down in semantics, I suggest that a Clojure application (such as the game discussed in this post) is something that can be easily extended using Clojure’s data structures and concurrency primitives. This may or may not be true of the hypothetical Clojure-wrapped game engine; it depends on the design and implementation details.

To explore what goes into making a game in Clojure, we’ll look at an implementation of Asteroids that uses Penumbra, an idiomatic wrapper for OpenGL. Penumbra was written to streamline doing interesting things with the graphics card, which includes small-scale game development. The issues I encountered while developing this game informed a number of design decisions I later made.

a description of the game

The game of Asteroids has three major elements: the spaceship, the bullets, and the eponymous asteroids. Optional elements include exploision effects for when an asteroid is split or the spaceship is destroyed, and exhaust effects to indicate thrust from the spaceship.

When implementing the game, we should first consider how each of these elements behave, as well as how they interact. Some of these details will vary between implementations, but for simplicity’s sake we will assume the following:

• Bullets travel a straight, predictable path.
  • If they come into contact with an asteroid, they blow up the asteroid and disappear.
  • If they don’t encounter an asteroid, they will eventually disappear.
• Asteroids travel a straight, predictable path.
  • Asteroids come in three sizes: large, medium, and small.
  • Asteroids explode when they come into contact with a bullet or the spaceship.
  • When a large or medium asteroid explodes, it emits four smaller, faster asteroids in random directions. Small asteroids disappear altogether.
• The spaceship travels an unpredictable path dictated by user input.
  • When the spaceship comes into contact with an asteroid, it explodes.
  • The spaceship has a position and velocity.
    • The position is constantly updated w.r.t. the velocity and elapsed time since the last update.
    • When the engine is off, the velocity remains constant. When the engine is engaged, the velocity is constantly updated w.r.t. the orientation of the ship and the elapsed time since the last update.
• Exhaust particles travel a straight, predictable path.
  • They are emitted whenever the ship’s engine is engaged.
  • They travel the opposite direction of the ship, adjusted for the ship’s velocity (this is important; things will look very strange otherwise).
  • They are purely for show, and don’t interact with anything.
  • After a while, they disappear.
• Explosion particles travel a straight, predictable path.
  • They are emitted in random directions from the explosion’s center.
  • They are purely for show, and don’t interact with anything.
  • After a while, they disappear.

Looking at this list, we can see that four out of the five elements have paths that are predestined; their position at any given moment is determined the moment they are created. This means that while we could store their position in memory and update that position each frame, we don’t have to. Instead, we have the option of defining a function which takes the game time as a parameter, and returns the current position.

This approach has two benefits. First, it’s much less taxing to the garbage collector; while a perfect garbage collector would be able to handle constant, predictable churn by just allocating over the previous frame’s state, reality is a little less kind. Updating too many things too often will lead to noticeable GC pauses, which any real-time application should strive to avoid. Second, it lets us treat the game as a constantly changing system which is sampled by the renderer, rather than a succession of snapshots whose frequency is arbitrarily dictated by the game’s frame rate. Specific reasons for why this is desirable will be discussed later.

It’s also obvious that the only difference between the exhaust and explosion particles are their initial conditions. As long as we make our generation method flexible enough, there’s no reason to deal with these elements separately.

So to create the game, we need methods for collision testing, generation, physics (for the spaceship), and finally rendering. Below, we’ll look at a few of the more interesting aspects. The complete source can be found here.

collision

We need to be able to test collisions between asteroids and ships, and asteroids and bullets. Bullets are perfectly circular, but asteroids and the ship are irregular shapes, so doing this accurately could be pretty complicated. But if we just assume everything’s a circle, it’s easy:

Notice that this code makes a number of assumptions about how the program will be structured: all of our game elements will be hashes, all of them will have :position and :radius keys, and those keys will either return static values or functions.

Nowhere do we define an interface that explicitly states this, nor can we tell at compile time whether the game elements actually satisfy these requirements. It’s easy to see how this approach could cause issues in larger scale development, but it does give us a certain flexibility, and for a program this size, the complexity is manageable.

generating asteroids

All of the elements exist on a 2-D plane, but that doesn’t mean we can’t render them as 3-D objects. We want our asteroids to be spherical, but irregular. We could randomly perturb the vertices, but that would just give us a spiky ball. The position of one vertex can’t be independent of its neighbors. In other words, we want to generate a fractal asteroid.

There are a number of different ways to accomplish this, but one of the simplest is the fault line algorithm:

• Split the asteroid into equal halves.
• Expand everything in one half, and shrink everything in the other.
• Repeat.

The resulting asteroid isn’t too distinctive, so we can just generate a few asteroid templates, and choose one at random whenever we create a new asteroid.

generating particles

Particles are soft-edged circles which, in large numbers, can be used to represent amorphous shapes. To create a texture, we define a texture which is perfectly white, but with varying transparency. Towards the middle, it is perfectly opaque, but falls off towards the edges per a Gaussian function.

which yields

Notice that we’ve wrapped the definitions of particle-tex and particle-quad inside a function. This is because we need to have both of these executed inside an OpenGL context. This allows us to freely reference these vars in the rest of the code, as long as we make sure init-particles is executed within an OpenGL context, and before any code that references the internal definitions.

Particles can fill a number of roles. Individual particles can represent bullets, and large quantities can be used to represent both explosions and the flame from the ship.

updating the game

Every frame, we need to determine the position of the various game elements, and draw them. We also need to update these elements – testing for collisions, generating particles when necessary – but this is a completely orthogonal concern to the rendering. Most games are single-threaded, and therefore the sequence of events is something like this:

• Test for user input, and change any related values.
• Update all the elements in the game, taking into consideration the elapsed time since we last updated the elements.
• Render the newly updated elements.
• Repeat.

This artificially conflates the act of updating and rendering, but with Clojure, we’re not nearly as constrained.

In Penumbra, an application is defined as a series of callbacks, such as :init, :mouse-down and :display. With the exception of :display, which cannot affect the game state, these are pure functions, which take the current state of the game, and return an altered version of that state. As a result, each function is independent of the others; as long as they don’t execute at the same time, it doesn’t matter what thread it’s executing on.

These callbacks are called in response to user input (:key-press, :mouse-up) or a frame being rendered (:update, :display). However, how often we test for collisions shouldn’t have to be coupled to how often we render, nor should how often we emit particles for the spaceship’s exhaust. In fact, in the latter case this would only make things look strange: if our frame rate stutters, so would the exhaust. Our rendering should just be a periodic glance into the world of the game, not something that dictates its appearance.

Penumbra supports periodic updates, which are defined as pure functions which alter the state at regular intervals. We can test for collisions 10 times a second, and emit exhaust particles 50 times a second, regardless of the game’s frame rate. This is used somewhat trivially in this game, but it’s worth noting that this can be an extremely powerful capability: we can reason about the game as a collection of independent, asynchronous processes, without concern for the underlying implementation. In this Tetris implementation, for instance, the descent of the blocks is controlled by a periodic update which alters its own frequency based on whether the down arrow is pressed.

Brian Carper recently posted about his difficulties using Clojure’s concurrency primitives during his implementation of an RPG. While I don’t think the approach used by Penumbra is the only way forward, at the very least it demonstrates that it is quite possible to leverage Clojure’s unique capabilities towards the development of games or other real-time graphical applications.

I have recently released version 0.5.0 of Penumbra to clojars, which should greatly simplify its use in a separate project. If anyone has any questions regarding how they could use it for their project, I’d be happy to help.

in real life

In memoriam of when I had the time and energy to write fiction.

***

The members of the Dadaist forums discuss the life and times of Marcel Duchamp, so I asked them for help with my essay. A guy named ManRay asked what I needed. 

All I’ve got so far, I said, is “Marcel Duchamp was born on July 28, 1887, in the town of Blainville, France.” 

Well, he said, there’s your problem. It’s not nearly long enough. You should work on that. 

# 

When the library closed, I walked back to the bus stop. A man was lying on the bench, his body dimly lit by the yellow streetlamps overhead. He wore a jacket patched at the seams with duct tape, and mumbled quietly to himself. His voice was discontinuous, halting suddenly and then resuming as though the silence had never existed. I stood beside him, watching cars pass by, waiting for the bus to arrive. 

When the bus came, he sat up. I walked on and he followed, digging in his pockets for change. The bus was empty, and I sat near the back. After he was done counting out the fare, he walked back and sat across the aisle from me. 

“Nice bus,” he said. 

My window had been tagged dozens of times, the sharp, jagged lines hopelessly tangled together. 

“Yeah,” I said. “Nice.” 

He leaned his head against his window, and stared at the passing buildings. 

“The thing about buses,” he said, “is that you don’t have to do anything. You just sit there, and the world slides on past you.” 

# 

• Marcel Duchamp was born on July 28, 1887, in the town of Blainville, France. For a time, he lived. He died in Paris on October 1, 1968. 
• Marcel Duchamp was born on July 28, 1887, in the town of Blainville, France. When he was twenty-six, he fell asleep and began to dream of the world we live in. When he wakes, we will all disappear. 
• Marcel Duchamp was born on July 28, 1887, in the town of Blainville, France. Napoleon Bonaparte died on May 5, 1821, on the island of Elba. Felix the Cat was seen on November 22, 1927, in the Macy’s Thanksgiving Day Parade. 

# 

When Duchamp wrote, he liked to use prime words, which are only divisible (definable) by themselves. It wasn’t easy. He had to invent them all. 

# 

I asked on the forums how I could best sum up Duchamp’s contribution to art. 

There weren’t any contributions for him to make, ManRay said. Or anything to which he could contribute. 

I don’t get it, I said. 

What’s to get? he said. 

# 

I couldn’t think of anything to do that afternoon, so I went to class. As the professor lectured, he drew uneven shapes which skittered across the chalkboard and crowded up against each other. 

“As we discussed last week, our baseline for realism is dependant on the media and genre. If enough media is created using a particular aesthetic code, then that code will begin to define the norms of the reality in which it exists. 

“For instance, if all movies involved ducks hopping across the screen on one leg, what would strike us as meaningful about each film is not the ducks themselves, but the variations in the style and tempo of their hopping. The ducks would, in effect, become invisible.” 

The girl sitting next to me wrote “invisible ducks” in her notes, and underlined it twice. 

After class, I walked up to the professor. “I need help with my essay,” I said. 

“Well,” he said, “Marcel Duchamp was born in Blainville, France.” 

“So I’ve heard,” I said. 

“And the thing to understand,” he said, drawing broad, spiraling figures on the board, “is that he never meant the things he said, nor did he mean their opposite. His true meaning hung in the middle like . . .,” he trailed off and stood there, the chalk still pressed against the board. Its tip began to crumble, and white dust fell slowly to the ground. 

Finally he turned to me. “Well,” he said, “like something. You know.” 

# 

The twenty-two inch bottle brush which hangs off of Duchamp’s painting Tu M’ represents: 

• his latent homosexuality 
• a sublimated desire for his sister 
• this one time when he was trying to clean the grout behind his toilet but couldn’t quite reach it 
• a penis 

# 

ManRay messaged me to say that someone on the forums was throwing a party nearby. 

I can’t go, I said. I’ve got an essay. 

Think of it as research, he said. You should never pass up an opportunity for research. 

# 

• Duchamp once said that to live is to believe. That was his belief, at any rate.
• “To live is to believe,” Duchamp once said. “That’s my belief, at any rate.” 
• To live is to believe. That was Duchamp’s belief, at any rate. 

# 

When I walked into the coffee shop, I passed by a girl who was sitting in an oversized armchair, balancing a laptop precariously on the edge of her knees. It was the sort of computer that would have been bought by parents with more money than common sense, and she had a cute little frown on her face as she tapped the keyboard in frustration. 

A man sitting at a nearby table was pouring sugar packets into his coffee as he stared at her. His hair was short, and noticeably receding. As he asked her if she needed any help, he ran his hand through it self-consciously, lingering on his bald spot. 

She pointed to her computer. “It isn’t working,” she said. 

He asked if he could look at it for her, and she handed it to him. He took a small sip of his coffee as he looked at the screen with furrowed brows. After a moment, he looked up at her. 

“What exactly’s wrong with it?” 

“None of the websites will come up,” she said. 

He nodded sagely, and began to work away at the computer. 

"If this ever happens again,” he said, “you should open up this dialog box and click here and here.” He poked the screen with his finger. 

The computer still didn’t work. He cleared his throat and kept on talking. 

She idly stirred her coffee with her finger, and made small sounds of encouragement and comprehension. When she began to stare out the window, he didn’t notice. 

Finally, he passed the working computer back to her. “Let me give you my card, in case you need any more help,” he said, rummaging around in his bag. 

“That’s alright, I think I can handle it,” she said, a bit too quickly. 

He pulled his hand out of his bag as if burnt. He stood up, and said he had to go. She nodded without looking at him. As he backed away from her, he stumbled over a chair. I followed him outside. 

“Do you want to go now?” I asked. 

He looked at me for a moment. “Sure,” he said. 

# 

When I walked in the door of the house, I didn’t see anyone at first, except for two guys sitting on the couch, staring at the opposite wall. There were four or five people running around the back yard and shouting, so I went out there to see what was going on. As soon as I walked outside, one of them ran up to me and punched me in the face. I held my face and stared at him. 

“What the fuck did you do that for?” I yelled. I felt like I might cry. 

“We’re playing John Calvinball. You were in the predestined violence zone,” he said. “It was inevitable.” He smiled at me, and a few others laughed. I walked inside, and sat down on the couch. ManRay handed me a tortilla chip. 

“Don’t worry,” he told me. “It’s a common beginner’s mistake.” 

“Oh,” I said. 

Someone offered me a drink, which he said was going to be a vodka tonic except that after he poured the vodka he didn’t have any room left in the cup. I took it. 

# 

The television was on, and we were watching an infomercial about a new technique for living our lives that would unlock our true potential when a guy walked in front of us and tripped over his own feet. He tried to get up, but his legs buckled underneath him and he fell again. 

“Are you okay?” I said. 

“This one time, Marcel Duchamp tried to sell toasters to the Amish,” he said. 

“Story of my life,” ManRay said. 

“He goes down to Pennsylvania, right, and walks into a town. He sees this guy driving a cart, and he’s got the beard and the hat and everything, so Duchamp walks up to him holding up a toaster in one hand. But the guy doesn’t even stop, and almost runs him over. 

“But Duchamp just figures the guy didn’t see him, and he walks up to the nearest farmhouse. He knocks on the door, and a man opens it, and he has the beard and hat, but he also has a gun. So Duchamp walks back onto the road, and tries the next house.” 

I walked over and tried to help him up. As he lay there, talking about how Duchamp couldn’t even sell a toaster when he offered some raspberry jam, a fifty dollar value, theirs for free if they ordered now, I saw that his pupils were the size of thumbtacks. 

“He doesn’t look so good,” I said. “Maybe we should take him to the emergency room.” 

“I’d take him, but my keys are in my pants,” ManRay said. He was sitting in a reclining armchair, wearing a t-shirt and boxers. 

“So where are your pants?” 

“Got me,” he said. “I was wearing them on my head for a while after I walked into the eclectic haberdashery zone, but I don’t remember what I did with them after that.” 

I walked outside and found the pants lying on the lawn. I stepped over the guy on the floor, who was staring at the blinking clock on the VCR and mumbling to himself, and handed them to ManRay. 

“Thanks,” he said. “But let’s wait until the infomercial is over. I want to see how it ends.” 

# 

We pulled up in front of the hospital and I opened the door and told him to get out, but he just sat there, staring at the dashboard. I pulled him out the door one leg at a time, and then had him lean on me so that he didn’t fall, and as we walked towards the hospital he was saying, “The Amish don’t have hotels or anything, so when it gets dark Duchamp sneaks into a barn high up on a hill. He finds a pile of hay, and he’s lying there, staring at the ceiling, when all of a sudden he hears voices. So he hides, just as two men walk in and grab pitchforks. He watches them go, and through the half-opened doors he sees a bright light off in the distance. He sits there for a while, crouched behind the hay, but finally he decides to follow them. 

“As he gets close to the light, he realizes it’s a bonfire, and that the Amish are circled around it. And when he gets even closer, like right next to them, he sees that there are five stakes around the fire, and that there are people tied to them, and that they don’t have the beards or the hats. And then the Amish start to dance around the fire, circling the prisoners, spinning so fast that their hats are falling off.” 

The door automatically opened in front of me, and I had to drag him inside because he wasn’t even trying to move his legs anymore. The receptionist asked me what was wrong with him, and I said that I didn’t know, so she told me to sit down in the waiting room. I pulled him into a chair, and he slumped over, and I sat down and waited until a nurse walked up to us and started pulling up his eyelids.

And the nurse was saying, “Do you know his insurance policy number?” 

And he was sitting next to me, saying, “The Amish are dancing, whirling silhouettes against the fire, and the prisoners are screaming, but that only makes them dance faster, moving to a beat that only they can hear. And Duchamp, he just sits there.” 

And the receptionist was saying to the guy who just walked in with a piece of sheet metal stuck in his arm, “On a scale from a little to a lot, how much does it hurt? Somewhere in between?” 

He was resting his head on my shoulder and speaking even faster now, slurring his words and vaguely grasping my arm. A doctor came, and put him into a wheelchair. As he was being taken away, he turned and looked at me and said, “He never tried to sell a toaster ever again.” 

I walked up to ManRay, who was talking to the receptionist about the computer she was using, and told him that we could go now if he wanted to. He looked at me imploringly, and asked if I didn’t want to wait just a little bit longer to see if that guy was alright and stuff. I started reading a pamphlet on the patient’s bill of rights. When he offered her his card, she smiled weakly and took it. He kept talking. I decided to call a cab. 

I was waiting outside when the cab arrived, wandering aimlessly around the empty parking lot. 

“Have some sort of emergency?” the driver asked. 

“I guess,” I said. 

“Well, don’t worry about any of that right now,” he said. “Just sit back and relax.” 

The meter counted off quarters as the streetlamps slid past my window. 

“Yeah,” I said. “Okay.”

précis

I haven’t posted anything in a few months. Here are a few things I’ve done with Penumbra in the interim:

• Switched from JOGL to LWJGL. OpenGL is a purely procedural library, which has specific threading requirements. JOGL tries to abstract away both of these facts, with only middling success. With LWJGL, development at the REPL becomes much more straightforward.
• Wrote versions of Pong, Tetris, and Asteroids.
• Further support for GPU programming. Wrote an implementation of Brian’s Brain which runs orders of magnitude faster than some other implementations (though, of course, that’s not an apples to apples comparison).
• More documentation. This is an ongoing effort.

I’ll be writing some more detailed entries in the near future.

exploring the mandelbrot set with your gpu

The Mandelbrot set is a good candidate for GPU implementation: it's simple, uses vector math, and is embarrassingly parallel.  Doing so, however, generally adds a significant amount of complexity to a program.  The programmer is forced to twist all computations to fit within the graphics pipeline, and a significant amount of ceremony surrounds even the simplest operation.  

performance in factor, java, and clojure

Slava Pestov recently posted a performance comparison between Factor and Java, based on a heavily numerical benchmark.  I've recently added support to Penumbra for offloading computation to the graphics card (GPGPU), which is ideally suited to this sort of computation, so I decided to give it a try.  It's not exactly an apples-to-apples comparison, but the results are interesting nonetheless.

using shaders in clojure

The graphics card is a massively parallel processor.  It processes graphics first at the geometry level (per-vertex), and then at the image level (per-pixel).  These two steps are called the vertex shader and fragment shader, respectively.  These are normally specified in a C-like language, but Penumbra uses an intermediate s-expression representation, which is translated into GLSL at runtime.

The above shader, with 'noise' equal to 0, looks like this:

This means that we don't have to use a simple shape like a square.  We can use any shape we like, and the grains will travel a continuous path along the z-axis.

Complete code can be found in the /examples subdirectory of the Penumbra repository.

rendering textures in clojure

At its simplest, a texture is an image stretched across a shape. We can define a texture like so:

(def checkers (create-color-texture 128 128))

(draw-to-subsampled-texture 
  checkers 
  
  (fn [[x y] _] 
   
    (if (xor (even? (int (/ x 16))) (even? (int (/ y 16)))) 
   
      [0.8 0.1 0.1 1] 
   
      [0.1 0.1 0.1 1])))) 

 
(defn textured-quad [] 
  (push-matrix 
   (translate -0.5 -0.5 0.5) 
   (normal 0 0 -1) 
   (draw-quads 
    (texture 1 1) (vertex 1 1 0) 
    (texture 0 1) (vertex 0 1 0) 
    (texture 0 0) (vertex 0 0 0) 
    (texture 1 0) (vertex 1 0 0)))) 
 
(bind-texture checkers) 
(textured-quad) 

We can apply the same texture to multiple shapes. Wherever a textured quad is drawn, the texture that is currently bound will be drawn across it.

 
(defn textured-cube [] 
  (dotimes [_ 4] 
   (rotate 90 0 1 0) 
   (textured-quad)) 
  (rotate 90 1 0 0) 
  (textured-quad) 
  (rotate 180 1 0 0) 
  (textured-quad)) 
 
(bind-texture checkers) 
(textured-cube) 

This rendering of a cube can itself be made into a texture, which can be applied to another shape. 

 
(def scene (create-texture 128 128)) 
 
(render-to-texture scene 
 (with-projection (frustum-view 50 1 0.1 10) 
  (textured-cube))) 

Putting everything together, we get this:

These examples use the Penumbra library. Full code can be found in the /examples subdirectory.

the alchemist