Straw Dogs

For those not knowing what I'm on about: Mechanize is a stunning little library which is great for automating web navigation and is especially suited to handling forms.

The latest version addresses a couple of bugs and is available here: Mechanize.

Well yesterday saw the release of the much anticipated Django 1.0.  Many years in the making its been well worth the wait.  Now although there's been numerous changes I've condensed this to the top 5 changes:

1. Django Comments

Although this has been available in some form for a while it's recently undergone major changes during the Google Summer of Code which has resulted in a very powerful, yet flexibile comments platform for your Django app.

Comments Upgrade Guide

2. Refactored Admin Control

The admin area has had two major changes that will see a much cleaner and flexible control interface overall.  Firstly is the latest newforms code bveing integrated into the admin.  Secondly is the decoupling of models from admin interface.  So gone are the days of subclasses within the models of your application.

Django Admin Reference

3. Data Query Changes

Django's ORM has had some major additions and clearup recently with some nifty features added including:

• values() - This will get a list of related values from a table linked by one-to-many or one-to-one.  Returning this as a dictionary it can be useful for constructing selection boxes.  A similar feature to this exists in CakePHP.
• values_list() - As above but returns a tuple instead of a dictionary.
• update() - This method was added to enable updates of a value across multiple objects.

Query Set Changes

4. GeoDjango

I have absolutely no idea what this really is but everyone seems excited so I am too!!!!  Woooohoooo!!!!

GeoDjango Info

5. Unicode

The average user won't care much about this but trust me when I say that the refactoriungs done with regard to unicode support will have a great impact.

Django and Unicode

Its easy to forget some of the best discoveries are made simply by browsing RubyForge.  This is now exception.  Its a nice little Ruby Gem to help Vim users to compare and manage their swp files.  These are produced as you're editing a file and so if the session dies or there's a crash then the swp is left behind for you to recover.  Problem being its not easy to determine the difference between the saved file 'proper' and the swp file.

That's where this little gem comes in handy: vimrecover.

P.S. Sorry for overusing that damn Ruby logo but I dunno what else to do.

If you're wanting to develop a Python GUI anytime soon can I strongly suggest you re/learn Java. I recently wanted to produce a simple yet functional desktop app and decided to give Python a go. I'd later py2exe my creation to create an executable Win32 app. My thinking was that Python is much faster to develop with and so doing a simple GUI app using it must be simple too.

Oh how wrong I was.

Coding the GUI

First off I tried the oldschool method of developing the GUI by hand using Tkinter as it seemed to have the greatest level of documentation and it was the library used in the Programming Python book. Alas this brought back painful memories of oldschool Java GUI programming with extremely restrictive rules on placement and positioning. After 3 hours of trying to get exactly what I wanted I ended up with GUI cat-sick on my screen - not good.

I tinkered with a couple of ther other libraries wxPython, PyGtk and even EasyGUI (too basic) only to hit the same problem - lack of flexibility.

I decided to try a different approach and find a GUI capable of handling the graphical side of things for me - maybe its a cheaters way out but personally I wanted an app not an education.

PythonCard

Oh dear god in heaven above. This app has been getting developed in a cave by the most hardcore, life-starved, miserable geekjects on the planet. It's based on an age-old Apple Mac application for developing quick dirty GUI apps. Which is great except its method of working is completely different to any other IDEs out there. It does follow a convention - its just the convention died years ago. That doesn't deter the developers though and they still have arguments to this day about wether to make tab ordering an easier to access feature or just leave as is (NB: Tab ordering is determined by the ordering of the elements in the frame and so you use "send to back", "bring forward" commands - very intuitive).

Boa Constructor

You need a book to operate this and also colour blindness helps because the interface is...well....vile! It's a painful mess to use and I didn't even bother trying - I closed it down in disgust. End of story.

IronPython Studio

You need a full version of Visual Studio - no go.

PythonWorks

Project now dead.

PAGE (Python Automatic GUI Generator)

This is one hell of a piece of wonder. Its amazing the sub-sorry I can't be sarcastic any longer. It's rubbish, it performs a role and thats it. It simply generates your GUI code rather than being a full GUI IDE.

I eventually got onto Glade and there's a number of options here that Python can exploit however, I'd wasted far too much of my valuable time by this point so I decided to do the only sensible thing:

Remember My Java

I left Java many moons ago but from when I'd last used Netbeans I remember the GUI building facilities were absolutely stellar. I wasn't disappointed this time round either. I can't believe how far along its now come and developing a GUI app in Netbeans using the Swing and SwingX components is an absolute joy. Its making me want to make more and more - I've restarted learning Java and not looking back.

So if you want to create a GUI desktop app - don't use Python (or Ruby).

PyConUK is just around the corner and the early bird tickets are only on sale until August 4th so be sure to get your tickets soon.

You can get the latest on the talks here: PyCon Talks

The Friday schedule of tutorials is available here: Tutorials.

This year Jacob Caplan Moss will be doing the Django tutorial!

The UK Python Conference is based in Birmingham UK and the website is here: PyCon 2008

I've tried more than once to jump onto the Google App Engine bandwagon to no avail. Unfortunately it's preview nature is acting to frustrate any attempt I make build something I'm happy with. Just some thoughts so far:

No Direction - It's been dumped into the ether with no solid direction. No recommended method or path to building your web app is given or hinted at. I'm all for choice and its great Google allow you to use Django, CherryPy, WebPy, etc. but it'd be helpful if a consistent and solid method was given. Unfortunately you're left with the feeling that the GAE has been thrown together with no real thought of making a web app out of it.

Disorganised Documentation - This doesn't help the problem mentioned above as the docs provide 3 or 4 ways of producing a web app from the start. It first tells you how to do it using basic CGI and printing directly to the browser. Then it shows you using the RequestHandler and WSGI app. At which point it doesn't say URLs are now handled by the WSGIApp instead of the app.yaml. Then it brings Django templates into the mix. How about pretty URLs? Oh well we can do that in app.yaml, or using matched groups in the WSGI app. For the love of all thats holy can't we just have a single nice solution? A recommended method? An idea? A guide?

Restrictions - So it gives you all that wonderful choice as mentioned above so you think you're going to make the best of it but oh no. Now you have to put up with the restrictions. No MySQL, no Django admin, restricted URL retrival, restricted security, no file uploading.

So you have to ask the question: Why?

What, ultimately, is the point in it when there's far better out there - Django, Merb, etc.  It has no purpose other than to faff with and so I shall be avoiding it for the forseeable.

Well it's been a while since the PHP Asterisk Manager API was released and I've yet to produce a clear and definitive example of its use.  So here goes.

Prerequisites

Your Asterisk server must be set up correctly before this API can 'speak' to the server.

• First you'll need to ensure the Manager interface is active and you've set the correct permissions (If you'd like this explaining please leave a comment).
• Secondly make sure you are setup with the ability for two endpoints to call each other.  This won't work if you have one phone on the server.

Install

Firstly you'll need to get hold of the library and there's two routes to take.  You can either install via PEAR or simply checkout the source from the Google Project:

svn checkout http://asterisk-php-api.googlecode.com/svn/trunk/ asterisk-php-api

Originating a Call

Once you've got it all ready you need to start your new file thats going to make use of it.  In this example I'm creating a simple dialler called: "call.php" which will take two GET variables, with one being the callee and one the caller.

 
<?php
/**
* To use this simply call it directly or via an Ajax call.  E.g.
* http://192.168.1.5/call.php?no=07943233397&from=229
*/
 
require 'AsteriskManager.php';
 
$number   = str_replace(array(' ', '(', ')', '.', ','), '', $_GET['no']);  //I'm just cleaning the input here.
$from      = 'SIP/'.$_GET['from']; //The SIP handset of who's dialling the outside line
$context  = 'asteriskcontent'; //This is the context in your Asterisk dial plan that this call is made to
$cid        = $number;
 
//Setup basic parameters for the connection
$params = array(
  'server' => '192.168.1.5',
  'port' => '5038',
  'auto_connect' => true
);
 
//Initialise
$am = new Net_AsteriskManager($params);
 
//Login to the Asterisk Manager interface
$am->login('user', 'pass');
 
//Originate Call is one of the commands available and it causes a call to be made and connected between two endpoints.
$am->originateCall($number, $from, $context, $cid, 1, 30000);
?>
 

By looking at the comments above you should see how this works and the steps required. If not then leave any comments and I'll build on the above. It's sometimes difficult to know what people already know so apologies if this is too simple or complex.

Later on I'll show how to work with queues - adding, removing, listing, etc.

Where are you on this matrix?

I've just taken up using Twitter.  So far so good.  Two points though, one of which isn't the fault of Twitter but:

1. DowningStreet doesn't bloody shut up!!

2. The site has complained about being snowed under 3 times so far.  They could sort this easily by using Django instead of Rails.

Other than that it seems alright.  For Rubyists you may be interested in the Twitter Rubygem by John Nunemaker: Twitter.


gem install twitter


P.S. Green as in: new, rookie, etc

I've always been a bit uncomfortable with the claims of 250 req/sec for this and 500 req/sec for that. It's never felt quite....right.  Or real.  After all how many sites truly get that many concurrent requests?  How many visitors a day would you need to get spikes of 500 req/sec hitting your server?

Well it seems the metric is flawed and so sayeth one of the developers of the Google App Engine during one of the IO talks done in May.

In that talk he mentions that people have been trying to run load tests on the GAE that simply throw hundreds of concurrent requests and conclude the test after one second.  Unfortunately this isn't realisitic and so the optimisations that are built into the system don't have time to kick in.

I'd say this isn't just an issue for GAE - its an issue with this benchmarking method full-stop.  Because as he says - its unrealistic.

Take the following example numbers from the slides:

Realistic values
• 50,000 users / day
• 2 pageviews / user
• 100,000 pageviews / day
• 5 requests / pageview
• 500,000 requests / day
• 5.8 requests / second

To some people 50k users may not seem much but irrespective I'd say its certainly a sizable chunk for the majority of readers here.  As you can see if a site is being hit by 50,000 users per day they can expect a realistic hit rate of 5.8 concurrent requests per second.

Let me say that again: A site that gets 50,000 user per day can expect just 5.8 average concurrent requests per second.

Thats a realistic number and it blows out of the water the benchmarking methods that hammer a server for one second with hundreds of requests.  People using those benchmarks have two flaws:

1. If they get a low number they may be unfairly assuming a poor service based on a spurious and unrealistic testing methodology.
2. If they get a high number they'll assume that their server can take sustained heavy load which is certainly not the case.

What the speaker then recommends is performing more realistic load tests spread over time.  This not only provides a more realistic basis for conclusions but also enables servers to adapt to the increasing load.

You can see the video which is about making production ready apps on Google App Engine by visiting its IO page here.

UPDATE

Digg Effect / Slashdotting

One of the obvious counters to this criticism which is also expressed at the end of this talk is that the Digg Effect or Slashdotting is indeed something that requires high 'per second' load testing.  However, thats not the case as these phenomena actually occur over a period of minutes and even hours and not a vertical spike.  The number of concurrent hits is still relatively low and so these hundreds per/sec tests don't fit here either.