Archive for June, 2012

Thursday, June 21st, 2012

The last post ended with a link to a Python game.  In the talk I gave introducing Python there was a lot more delving into what the actual game did, but it was a more interactive approach were there could be more dialog on how the intro slides were actually applying to the small snake game.  Blogging does not lend itself to that same sort of interactivity.

However blogging does allow me to actually break down the game in a way that would have just run out of time during the live talk. Which is what I did in this entry.

Here is another slide from the talk I gave:

Slide17

This slide was not my own creation.  MikeD helped me explain the actual high level architecture of a simple video game.  The snake game itself mostly follows these principles.  I will refer back to the diagram on the slide during most of the rest of this entry.

Again here is the full source: https://bitbucket.org/sirchristian/snakes and the version of the code that this entry talks about specifically can be found at the ‘presentation version’ tag here: https://bitbucket.org/sirchristian/snakes/src/083c6db7b76e

Start

The main file for the snake program is ‘python.py’ (aside: naming your python file python.py really makes tab completion more difficult than it should be). Open that up and you will see the first thing that the Python interpreter will execute.

import pygame
from random import randint
from myobjects.snake import snake
from myobjects.food import food

# Settings
SCREENSIZE = (1024,768)
BGCOLOR = (0xff, 0xa5, 0x00)
FRAMERATE = 30

The first few statements are telling the interpreter what other libraries the program will need to use. The two recommended ways to include other libraries are using the ‘import x’ statement which will import the whole module/package and the ‘from x import y’ statement which lets you pick specific items to import and import them directly into the namespace of the module importing them.

The second block of code is defining what the program will use as constants. Note: the constants are only a convention in Python.

if __name__ == '__main__':
playGame()

The next code that will get run is at the bottom of the file.  Everything in between the imports/constants and the block of code above is a function definition. The functions are loaded into the namespace, but they are not executed. One thing to notice in the above code are ‘__name__’ and ‘__main__’. Identifiers that start and end in double underscores is a convention that says they are special Python symbols.  In this case all we are checking to see is if the current file is the file that was passed directly to the interpreter (not imported from anywhere else) then if it is we will execute the playGame() function.

Load Assets

Next let’s look at the playGame function.

def playGame():
# define the arrow keys
arrow_keys = (pygame.K_UP, pygame.K_DOWN, pygame.K_LEFT, pygame.K_RIGHT)

# init pygame
pygame.init()
game_surface = pygame.display.set_mode(SCREENSIZE)
# set the key repeat speed
pygame.key.set_repeat(10, 25)
# set the caption
pygame.display.set_caption('SNAKES!!!! - Press <Esc> to Quit, <R> to Restart')

The code above is the first part of the playGame function. The first thing playGame does is initialize everything needed to play the game. Most of the hardware assets get loaded/initialized during the pygame.init() call. There is a game surface that is being created. By default pygame disables key repeating so pressing a key and holding it gets counted as 1 key press. This functionality is not desirable for the snake game as the controls for moving the snake will be the arrow keys. Most people will be used to just pressing and holding the arrow keys to create motion. The last bit if initialization is setting a caption for our game window.

Game Loops(s)

In the snake game there are 2 nested loops.  The outer loop is a ‘replay’ loop. This allows for the game to be played multiple times (the game does not allow the user to actually win, so allowing a replay at least makes losing more fun). The inner loop is what is more like a typical ‘main game loop’.


replay = True
while replay:
replay = False
# ... snip ...
# objects that need to be instantiated
# ... end snip ...
game_clock = pygame.time.Clock()
playing = True
while playing:
game_clock.tick(FRAMERATE)

That is the main outline of the game structure. Nothing too fancy there you can see the main playing loop nested inside a replay loop. There is some extra initialization that goes on between the two loops. If you go to the game on bitbucket the objects that are instantiated are the 2 snakes that are playing against each other [the ‘good’ snake is the python (obviously) the ‘bad’ snake is a rattle snake].

There is also some clock logic going on in there. This is the game clock. Its not strictly necessary to use it, but the human brain/eyes are limited to seeing things 30 times per second (generally). Rather than chew up CPU and other resources displaying things that nobody will see we just tick the clock 30 times a second.

Artificial Intelligence

Artificial Intelligence in terms of gaming is where the computer figured out what it needs to do. The simple snake game using the word ‘intelligence’ is a stretch.


# always move the bad snake
rattle_num_frames_in_dir = rattle_num_frames_in_dir + 1
moved_rattle = rattle_num_frames_in_dir % 2 == 0
while not moved_rattle:
if rattle_num_frames_in_dir > rattle_max_frames_in_dir:
curr_dir = rattle_dir
while curr_dir == rattle_dir:
rattle_dir = arrow_keys[randint(0,3)]
rattle_max_frames_in_dir = randint(25,50)
rattle_num_frames_in_dir = 0
moved_rattle = rattle.move(rattle_dir,game_surface)
if not moved_rattle:
rattle_num_frames_in_dir = rattle_max_frames_in_dir+1

The rattle snake in the snake game is the snake that is controlled by the computer. Every frame we want to move the the rattle snake just a little bit. The rattle snake has very simple rules. It must move in the same direction for 25-50 frames, then change to a random direction. The rattle.move call will return false if the snake could not move for some reason (like it is trying to move off screen). Pretty basic. An interesting modification to the snake program would be adding some more logic that feels more like AI to the rattle snake (have the rattle snake actively try and eat the food, or actively try to hunt the other snake would be cool, however I leave this as a later exercise).

Physics

Here is where things get pretty interesting from the point of view of making the video game feel like a video game. In the snake program there are 2 kinds of physics going on. The first is moving all the objects where the need to be moved in the frame. The second form of physics is the collision detection.

# detect collisions with the bad snake
for r1,r2 in ((r1,r2)
for r1 in python.get_rects()
for r2 in rattle.get_rects()):
if r1.colliderect(r2):
playing = False
replay = gameOver(game_surface)
break

This piece of code is demonstrating how to use generator expressions in Python. Which is closely related to list comprehension with Python. In Python syntax like this: [x for x in range(10)] will return a list. Replace the outer square brackets with parentheses and you get a generator object. Looking at the piece of code above there is a call to get_rects (which returns a generator object) for the python and a call to get_rects for the rattle. The calls are being used inside a generator expressions that will return a new generator object. That new generator object is what is being looped over. This allows us to check each rectangle of the python with each rectangle of the rattle for a collision.

If the concepts of list generation/generator expressions are new to you I recommend looking over that piece of collision detection code again. There is a lot going on there, but once you ‘get it’ there are some powerful things that you can do. In an earlier version of the code this piece of code was a complex nested loop. The new version I find must more readable.


# detect collisions with ourself
if python.head_hit_body():
playing = False
replay = gameOver(game_surface)
break

# eat
python.try_eat(food_items)
rattle.try_eat(food_items)

# replace food
create_food()

The next set of collision detections are mostly encapsulated inside head_hit_body and try_eat. Those calls are simple collisions detections that iterating over the objects and detect of there was a collision on each iteration.

The call to create_food is a stretch to call physics. It creates objects if needed. The interesting piece to note in terms of Python development is that create_food is technically a closure. The other thing I would like to call out is if you pull down the snake game, trying to replace the hard coded ‘2’ in the create_food function with something large (like 50) creates a very different game play. Trust me, try it!

Draw Frame

Here is where we get more into the technical aspects of a game. Everything we did above with moving objects, deleting objects and creating objects were done in the background. At this point in the frame the objects are all updated to the new state for the frame, but are not yet visible to the user. In order to do that we need to blit the objects onto the game surface. Then we need to update the display.

# update display & objects
python.update(game_surface)
rattle.update(game_surface)
for f in food_items:
f.update(game_surface)
pygame.display.update()

Every object we have has an update method that take the surface in as an argument. Those functions know what rectangles they used and can blit them onto the surface. Take a look at the food update() function.

def update(self, game_surface):
""" Updates the food """
rect = self.surface.get_rect().move(self.pos)
game_surface.blit(self.surface, rect)

All the update function does is make sure the rectangles are in position and blits them. The update function in snake is slightly more invoked, but not by much. While the code for this looks pretty simplistic, it is actually very important to get this right. Most times when developing your game (at least in pygame) when the objects aren’t moving, or aren’t behaving properly it is prudent to make sure that this steps is being done correctly.

Once the game surface is updated a call to pygame.display.update() will actually update the display that the user sees

Detect and handle user input

The snake game actually does all its event processing at the beginning of the main game loop.

# handle events
for e in pygame.event.get():
if e.type == pygame.QUIT:
playing = False
elif e.type == pygame.KEYDOWN:
if e.key == pygame.K_ESCAPE:
playing = False
replay = False
if e.key == pygame.K_r:
playing = False
replay = True
if e.key in arrow_keys:
python.move(e.key,game_surface)

The snake game handles a quit event (i.e. the user closed the game by using the ‘X’ on the game window) and a few keyboard events. If the escape key was his the game will exit. If the ‘R’ key was hit the game will replay. If any arrow keys were hit the python snake will be asked to move. The move function was already touched upon above during the AI. All it does is move the snake in a direction as long as it doesn’t go off screen.

Closing

That the end of the brief tour of game development with Python. I encourage you to download the snake game a play with it a bit. Once you are comfortable with the basic concepts pygame has much more to offer.

Sunday, June 10th, 2012

Last week I gave a talk on an overview of Python.  The talk was focused on being a very brief introduction into Python.  The talk assumed that the audience had little to no Python experience, but had a solid background in another programming language (in this case everybody had solid C# experience).

This is a rehash of that presentation.

Slide1

Introductory Slides

Slide4

The main point here is that Python is more than just what you download when you go to Python.org. At the core Python is just a programming language spec. A whole community and ecosystem has evolved around that spec.  The language itself brings with it a core set of values that are embraced by the community.  These values are summed up in the ‘Zen of Python’.


"
>>> import this
The Zen of Python, by Tim Peters

Beautiful is better than ugly.
Explicit is better than implicit.
Simple is better than complex.
Complex is better than complicated.
Flat is better than nested.
Sparse is better than dense.
Readability counts.
Special cases aren't special enough to break the rules.
Although practicality beats purity.
Errors should never pass silently.
Unless explicitly silenced.
In the face of ambiguity, refuse the temptation to guess.
There should be one-- and preferably only one --obvious way to do it.
Although that way may not be obvious at first unless you're Dutch.
Now is better than never.
Although never is often better than *right* now.
If the implementation is hard to explain, it's a bad idea.
If the implementation is easy to explain, it may be a good idea.
Namespaces are one honking great idea -- let's do more of those!
"

Slide4

To reiterate: at the core Python is just a language spec.  Because of this there are more than just one implementation of Python. 

  • CPython written in C and is the reference implementation of Python. This is what is downloaded from Python.org.
    • Note: Version 3 is NOT backwards compatible with version 2. Some version 3 features were back ported, but they are still different. As of this writing my recommendation for getting started in Python would be to download the latest 2.7 version.
  • IronPython written in C#. Targets the Microsoft CLR.  It was built to showcases the DLR.
  • Jython written in Java. Targets JVM.
  • PyPy written in RPython. RPython a subset of Python, but specializes in writing JIT compiled interpreters. This is the implementation that intrigues me the most.
  • Others: http://wiki.python.org/moin/PythonImplementations

Slide7

The talk didn’t go into great depths about setting up a development environment or best practices while developing in Python (for a great piece on that visit: http://mirnazim.org/writings/python-ecosystem-introduction/).  However I wanted to mention pip,virtualenv, and setuptools.  These tools will greatly reduce some frustration when dealing with different Python environments and different libraries available.

Language Overview

Basic Types


i = 1 # integer
l = 777777L # long
f = 99.9 # float
s1 = 'Hello' # string
s2 = "I'm a string" # string
b = True # boolean
n = None # null

These are the basic types of Python. Notice how to declare a variable, you just set it equal to something and the interpreter figures out what type is. The only other thing going on in the code above is you can see how a comment is written in Python.

Containers


# tuple
t1 = (1,2.0,'hi')
t2 = tuple(['iterable'])

# list
l1 = [5, 4.0, 'bye']
l2 = list('iterable')

# dictionary
d1 = {'one': 1, 'two': 2, 3: 'three'}
d2 = dict(four=4, five=5.5)

# set
sl = set(['iterable'])

Containers are types that let you store other items in them. The 4 containers tuple,list,dictionary and set are 4 commonly used containers in Python. Python has a lot of support for manipulating containers, and passing them around. They are central to many Python programs.

  • Tuple – An immutable sequence. Items are stored in order, but the sequence cannot be changed once initialized.
  • List – A mutable sequence. Like a tuple a list’s items are stored inorder, but items can be added, removed, sorted, etc.
  • Dictionary – A key/value pair. The key must be unique, but is not limited to what type it is. The only requirement is that the key is hashable. A dictionary’s items are not stored in order, they are stored in order of the hash of the key in order to make lookups faster.
  • Set – A hashed collection of items. Think of sets as an unordered list.

Operators

  • Math: + – / * // ** %
  • Bitwise: & ^ ~ | << >>
  • Sequence: + * [:] [] in
  • Comparison: < > != == <= >= is not and or

A lot of the operators should be familiar to users of other languages, particularly languages with C like syntax. I’ll just call a few out. The // operator is floor division (not a comment). The ** operator is the power operator (2**3 means 2 to the power of 3). The [:] operator is the slice operator. I am not going to go much into slices, but they are an important concept in Python. Basically a slice is a section of a sequence. Numbers can be used in the slice operator to specify what section of a sequence required.

Conditionals


# if-elif-else
x = 1
y = 0
if x < y: print('weird') elif x > y:
print('duh')
else:
print('odd')

# Expression
x = 1 if True else 100

The only real conditional construct in Python the ‘if’ construct. It can be used as a statement or as an expression. Using it a statement we can see how to construct a code block in Python. A code block follows a colon and must be indented using the same indentation for the whole block. This is what it meant when people say whitespace matters in Python. It helps to keep the source code looking clean.

Loops


# A while loop
while True:
print('Hello')

# for loops
for x in range(10):
print x
for name in ('Chris', 'Mike', 'Joe'):
print ('Hello {0}'.format(name))
for name, age in [('Chris', 'old'), ('Clayton', 3), ('Ashton', 1)]:
print ("{0} is {1}".format(name, age))

Two basic loop constructs. A while loop acts pretty much like it does in many other languages, while condition do code block. For loops are more powerful in Python than a lot languages with a for loop rooted in C history. During each iteration of the loop the loop variables are initialized to the next item. Notice how there can be multiple loop variables. Also notice how a variable is not restricted to a type. The type can change on each iteration as long as it make sense to the code block.

Functions


# Regular Function
def my_fun(name='Nobody'):
return 'Howdy {0}'.format(name)

# returns 'Howdy Nobody'
my_fun()

# Functions are are first class
vfun = my_fun

# Returns 'Howdy Function'
vfun('Function')

# Lambda Function Note: cannot contain statements (i.e. print;for;etc)
lfun = lambda greeting, name: '{0} {1}'.format(greeting, name)

# Funtional functions
map(vfun, ['A', 'B']) # returns ['Howdy A', 'Howdy B']
filter(lambda big: big > 10, (100, 1)) # returns (100,)
reduce(lfun, ('Hi', 'Chris', 'H')) # returns 'Hi Chris H'

Functions are the smallest building block for holding code in Python. They are also first class, which mean they can be set to variables and be passed around. This allows for some basic functional style programming in Python. Python has lambdas, however they are not full functions. Typically a lambda is great for a simple one liner, however anything more complex a function can be used. Also note that functions can be declared in functions, this again makes lambda’s a little less desirable then they can be in other languages.

Code Organization

Slide14

Like what was mentioned before functions are the smallest level of code building blocks. Functions can live inside functions, modules, classes, and packages. Modules can also contain classes; packages can contain modules and classes. In terms of actually implementing the organization the easiest way to think about it is a function is a function, a class is a class a module is the file (file name is the module name) and a package is the directory name. There is a special file __init__.py that allows for more control over the package (and technically __init__.py is required to exist inside a package, but may be empty). They organization can be more complex, but doesn’t need to be.

Cool Stuff: Generators


def get_odds(nums):
for num in nums:
if num % 2 != 0:
yield num

for odd in get_odds(range(10)):
print(odd)

Generators are a way to control what happens in each iteration of a loop. Those of you familier with C#: think IEnumerable. On each iteration of the loop the generator will be called and the next object will be returned. One key to thinking about generators is that code is not called until runtime.

Cool Stuff: Decorators


from datetime import datetime

class logit:
def __init__ (self, function):
self.f = function

def __call__ (self, *args):
print("+++ {0} @ {1}".format(self.f.__name__, datetime.now()))
self.f(*args)
print("--- {0} @ {1}".format(self.f.__name__, datetime.now()))

@logit # <--- This is the decorator def add(nums): s = 0 for num in nums: s = s + num print (s) return s add(range(50000)) # output: # +++ add @ 2012-06-08 14:33:43.687000 # 1249975000 # --- add @ 2012-06-08 14:33:43.700000
A whole blog series could be written about decorators. The one shown here was chosen to show some of the power of decorators, without all the complexity that could come with implementing your own decorator. The simplest way to think of a decorator would be a function that gets called before the function that it decorates is called. The decorator itself is actually in charge of calling the decorated function. This allows decorators to do things before the decorated function is called, after the decorated functions is called, or even not call the decorated function at all (say you wanted a function to only run on Monday's, you can do that with decorators).

Sample Program - Snake game

In order to illustrate some of the power of Python I created a simple game and put the source online. Feel free to browse the source and drop me a line with any questions/comments. A lot of what was covered in the presentation is available in the game.

https://bitbucket.org/sirchristian/snakes

image

Closing

This was meant to just whet your palate when it comes to Python development. Python has many more neat features that I encourage everybody to check out. Go download your choice of Python and start developing!

Reference

Friday, June 1st, 2012

In one of my last projects, where Microsoft .NET assemblies can be updated at runtime, sometimes we ran into an exception, “There is an error in XML document”. Even when nothing changed in reference to any part of the code dealing with the object being serialized, or the serialization code itself we would see this error.

I sat down to try and figure out why.

The exception was happening inside the Microsoft generated dynamic dlls for XmlSerialization. You can get the dll to stick around if you follow Hanselman’s instructions.  Once we can step into the DLLs we see the error is happening on a line like this


if ((object)(a_0_0) == null) Reader.Skip(); else method9_Add.Invoke( ...

The error is a null reference exception because method9_Add is null.  So looking at what method9_Add  actually is:


System.Reflection.MethodInfo method9_Add = type2_Item.GetMethod("Add" ...

So they are using reflection. The method they are trying to get is the Add method on a generic dictionary.  In particular for they are trying to get the Add method for a List<MyBizObject>. This fails because GetMethod’s execution context doesn’t know anything about MyBizObject.

This begs the question as to why it works normally.  Doing the same steps without updating the dll at runtime:


if ((object)(a_0_0) == null) Reader.Skip(); else a_0_0.Add( ...

Ah ha! Normally XmlSerialization doesn’t use reflection, a_0_0 is a real object. After doing some digging in reflector we find.

clip_image002[4]

A null assembly.Location will mark the type as dynamic and a dynamic assembly triggers XmlSerialization to generate reflection members. When a DLL is loaded with using  Assembly.Load  assembly.Location to be an empty string. Causing our issue.

In this particular case the fix was to stop using a strongly typed List<MyBizObject> and just use an ArrayList. This way the reflection code generated by Microsoft worked fine.