Using Lillypond to show the ‘sim’ notation in a score

How can you include ‘sim’ in your score ?

Summary

How to define the ‘sim’ notation when using Lilypond to engrave a score. This is a little off topic for my blog but I found it difficult to find this information myself so I’m hoping others will find it here.

Lilypond and Frescobaldi

Lilypond is a file format allowing you to define a musical score using simple text notation; Lilypond is also the program which takes that text notation and outputs a score in, for instance, Acrobat PDF format. It’s a pretty amazing technology.

In using Lilypond I’ve found it very useful to use the Frescobaldi sheet music text editor which provides support for writing files in the Lilypond format.

What I’ve done

My music theory knowledge isn’t great but I’ve really enjoyed using Lilypond and Frescobaldi to engrave a score for a relative of mine. It’s just an amazing technology and the output is beautiful.

It took a little time to get to grips with some of the ideas although the Lilypond manuals are very good. I did however find one little thing I couldn’t work out how to do and that is the subject of this, rather top heavy post.

‘sim’ notation on a score

Sometimes in scores you want the text ‘sim.’ to appear as shown below in the screen dump.

I looked all over the documentation and there just didn’t seem to be a way of including that. However I visited the Lilypond IRC channel (#lilypond@irc.freenode.net) which you can visit directly by going to the support page and paging down a little.

The good folk on there were able to point me in the right direction. By appending

_\markup{\large \italic "sim."}

to the note I wished ‘sim’ to appear under the right thing was done.

Just to make that a little clearer here are a few bars which include the annotation to create the ‘sim’

As you might imagine the

\large

only controls how large the ‘sim’ appear on the score and I fiddled around with this a little until I found an adjective which produced text that matched the rest of the score. There are a number of options for altering the size of text displayed as can be seen in the Selecting font and font size part of the Lilypond manual.

Crazy little thing called : SimpleHTTPServer

Crazy little thing called : SimpleHTTPServer

Summary

Python brings you a one line web-server !

Simplest webserver ever !

I was logged onto a tiny VPS I use sometimes and thinking it would be useful to have a web-server to help move a few files off it. The thought of installing a ‘real’ web-server seemed a bit much for what would five minutes of use so I started thinking about Python (of course !).

It turns out that you can fire up a (completely unsecured and rather wild-west) web server like this :

python -m SimpleHTTPServer

When you type that command the current directory becomes the root of a web-resource, accessible via port 8000 – rather than the default port 80 we usually use for HTTP, and any files, and sub-directories, in the current directory can be accessed like this

http://mylittlevps.kubadev.com:8000/thefileIwant.txt

and, for stuff in sub-directories, like this

http://mylittlevps.kubadev.com:8000/foo/bar/anotherfileIwant.txt

Don’t say you weren’t warned

There are of course all sort of circumstances where doing this would be a REALLY BAD IDEA ™ but all the same it’s nice to know it’s there if you need it !

optparse: simulating ‘–help’

optparse: simulating ‘–help’

Summary

Getting optparse to show a summary of script options under program control.

What do you use optparse for ?

optparse is a Python standard library for parsing command-line options.

optparse provides a relatively simple way of

* defining a set of command line arguments which the user may enter
* parsing the command line entered and storing the results of the parse

I need help !

By default optparse will respond to a either of :

<yourscript> -h

or

<yourscript> --help

by printing a summary of your scripts options.

What I learned today

In the script I was working on today I wanted to respond to the user not entering any argument at all by printing a summary of all options available (in other words as if they user had entered ‘–help’ or ‘-h’ as an argument).

There is a way you can do this but for some reason it’s not in the list of OptionParser methods shown in the documentation.

print_help()

If you call the method ‘print_help’ as shown in the code below optparse will respond by by printing the same text that would be shown if the user were to enter an argument of ‘–help’.

parser = OptionParser(description=desc, usage=usage)
parser.add_option(  "-i", "--inbox", action="store",  dest="inbox",
metavar="INBOX", help="Location of INBOX")
parser.add_option(  "-o", "--outpath", action="store", dest="outpath",
metavar="PATH", help="PATH to output csv file")
parser.add_option(  "-v", "--verbose", action="store_true",
dest="verbose", help="Show each file processed")

(options, args) = parser.parse_args()

if (options.inbox is None) and (options.outpath is None):
  parser.print_help()
  exit(-1)
elif not os.path.exists(options.inbox):
  parser.error('inbox location does not exist')
elif not os.path.exists(os.path.dirname(options.outpath)):
  parser.error('path to ouput location does not exist')

return options

Using Paramiko to control an EC2 instance

An example of using Paramiko to issue commands to an EC2 instance

Summary

An example of using the Python library Paramiko to ‘remote control’ an EC2 instance .

“Do that, Do this”

Recently I’ve been looking into the use of the Bellatrix library to start, control and stop Amazon EC2 instances (my posts about that are here and here).

Spinning off the side of that I’ve taken a look at the Paramiko module which “implements the SSH2 protocol for secure (encrypted and authenticated) connections to remote machines”.

There’s a good article on beginning to use Paramiko “SSH Programming with Paramiko” by Jesse Noller which I found very helpful but there’s enough stuff I had to change to deal with using EC2 and the controlling Python script running on Windows that I thought it would be worth recording my sample script.

Installing Paramiko

So first off I’d seen the comments about Paramiko maybe needing a special compilation step for installation to Windows but I’m pleased to say that’s not true, I downloaded 1.7.7.1 to my Windows Vista machine, did a quick…

python setup.py install

… and it all went very smoothly, just to be sure I tried out an import …

>>> import paramiko

… no problem.

Starting the EC2 instance

I now needed a server to talk to so I used Bellatrix to spin up a micro instance of Ubuntu like this :

python "C:\bin\installed\Python2.6\Scripts\bellatrix" start --security_groups mySecGrp ami-3e9b4957 mykeypair

The arguments you can see here are :

  • “mySecGrp” is a Security Group I’ve previously setup via the AWS Management Console;
  • ‘ami-3e9b4957′ is the AMI of Ubuntu 10.04 (Lucid Lynx); and
  •  ‘mykeypair’ is the name of a Key Pair that, again, I’ve previously setup via the AWS Management Console.

When you run that command you get an output that looks like this :

C:\Users\Richard Shea>python "C:\bin\installed\Python2.6\Scripts\bellatrix" start --security_groups mySecGrp ami-3e9b4957 mykeypair
2012-04-19 21:27:03,135 INFO starting EC2 instance...
2012-04-19 21:27:03,180 INFO ami:ami-3e9b4957 type:t1.micro key_name:mykeypair security_groups:mySecGrp new size:None
2012-04-19 21:27:07,657 INFO starting image: ami-3e9b4957 key mykeypair type t1.micro shutdown_behavior terminate new size None
2012-04-19 21:27:08,555 INFO we got 1 instance (should be only one).
2012-04-19 21:27:08,556 INFO tagging instance:i-f1234567 key:Name value:Bellatrix started me
2012-04-19 21:27:12,361 INFO instance:i-f1234567 was successfully tagged with: key:Name value:Bellatrix started me
2012-04-19 21:27:12,361 INFO getting the dns name for instance: i-f1234567 time out is: 300 seconds...
2012-04-19 21:27:34,173 INFO DNS name for i-f1234567 is ec2-10-20-30-40.compute-1.amazonaws.com
2012-04-19 21:27:34,173 INFO waiting until instance: i-f1234567 is ready. Time out is: 300 seconds...
2012-04-19 21:27:34,174 INFO Instance i-f1234567 is running

And the key thing here is that we now now have access to the host name of the EC2 instance we’ve just spun up:

ec2-10-20-30-40.compute-1.amazonaws.com

Talking to the EC2 instance

We’re now ready to send commands to our new instance. Making use of some of Jesses code I was able to write :

import paramiko
ssh = paramiko.SSHClient()
ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
ssh.connect('ec2-107-22-80-32.compute-1.amazonaws.com',
            username='ubuntu',
            key_filename='''mykeypair-ssh2-rsa.openssh''')
stdin, stdout, stderr = ssh.exec_command("uptime;ls -l;touch mickymouse;ls -l;uptime")
stdin.flush()
data = stdout.read().splitlines()
for line in data:
    print line
ssh.close()

Anyone who’s got this far can probably see what’s happening here, but just to be sure :

  1. having instantiated an instance of paramiko.SSHClient we’re able to use our private key file and the address of our EC2 server to start an SSH session.
  2. We then use the exec_command method to submit a string of commands and get back three references to files corresponding to : standard input, standard output and standard error.
  3. By reading through the standard output file we can print locally the output from the commands executed on the EC2 instance.

The Key Thing

As you can see to identify ourselves to the remote server we’re doing a key exchange. Our private key is ‘mykeypair-ssh2-rsa.openssh’. A point worth mentioning here is that generally I logon to EC2 instances using the excellent PuTTY . The private key files used by default by PuTTY are not in the same format as the ones required by Paramiko so as a result when I first tried this I found Paramiko fell over complaining my ‘key_filename’ argument was ‘not a valid dsa private key file’.

PuttyGen to the rescue

Well the great thing is that PuTTY actually comes with a tool PuttyGen which will import a standard PuTTY key file (foo.ppk) – you need to do ‘Conversions’ | ‘Import Key’ and then ‘Conversions’ | ‘Export OpenSSH Key’

Ubuntu Specific

Bear in mind that the way the SSHClient connect method is used above is suitable for an Ubuntu instance as it is by default however you can’t rely on all *nix instance working just that way.

Seeing the Output

Just to close out I’ll show you the output

12:39:45 up  3:12,  0 users,  load average: 0.08, 0.02, 0.01
total 0
total 0
-rw-r--r-- 1 ubuntu ubuntu 0 2012-04-19 12:39 mickymouse
12:39:45 up  3:12,  0 users,  load average: 0.08, 0.02, 0.01

Powerful Stuff

The combination of Bellatrix allowing you to spin up EC2 instances with a single command and Paramiko allowing you send arbitary commands to those servers is powerful stuff and I’m impressed at the work done by their respective developers. Of course Bellatrix can do ‘for free’ what I’ve used Paramiko to do here are part of it’s Provisioning commands but it was an interesting exercise for me to do my own version of that.

Bellatrix Provisioning Commands

A quick guide to built-in Bellatrix provisioning commands

Summary

A table of Bellatrix provisioning commands as at 1.1.2

Bellatrix Provisioning

As mentioned in my previous post Bellatrix allows you to start and provision Amazon Web Services EC2 instances.In this context ‘provision’ refers to the process of installing software, creating directories, changing file permissions etc.

The built in commands are documented here in the official doco but before I found that I’d ripped the cmds.py file apart and produced the following table. It’s a little more succinct than the official version so I think it’s worth publishing here.

Version Alert

For obvious reasons this type of thing is prone to Bellatrix changing. This page is valid as of the 1.1.2 version of Bellatrix.

Provisioning Commands

Command Description
apt_get_install Return the “sudo apt-get install” command
apt_get_update Executes apt-get update.
chmod Applies the chmod command
createSoftLink Creates a new soft link
copy Copy a file using -f so it doesn’t fail if the destination exists
create_django_project Creates a Django project
createVirtualEnv Generate a new Python virtual environment using virtualenv
executeInVirtualEnv Execute a command within a virtualenv environmen
flatCommands Given a list of commands it will return a single one concatenated by ‘&&’ so they will be executed in sequence until any of them fail
install_pip Install pip using apt-get install
install_nginx Install Nginx in Ubuntu using the repo they provide as described in http://wiki.nginx.org/Install#Ubuntu_PPA
installPackageInVirtualEnv Install a Python package into a virtualenv
mkdir Created a new directory. “-p” flag is used so the command generates the same result regardless whether the directory exists or not.
pip_install Install a Python package using pip
sudo Execute a list of commands using sudo
wget Downloads a web resource using wget

Bellatrix + Windows Users – Two things to remember

Bellatrix – Windows Idiosyncracies

Summary

Two things you might find useful if you’re starting to use Bellatrix on Windows.

What’s Bellatrix Again ?

Bellatrix allows you browse and control Amazon EC2 instances using simple commands on your local machine.

For instance :

bellatrix start ami key_name

run on your machine will launch an EC2 instance within Amazon Web Services based on the named ami (Amazon Machine Image) and allow you to contact it using the key_name you have specified.

You’ve been able to do something like this with the Python package Boto for some time but Bellatrix provides a lot of ready to use commands rather than having to write your own scripts.

Bellatrix on Windows Then ?

Location of Home

I’m a bit embarrassed to write this but in the Bellatrix doco which describes what config files you need and where to place them it refers to

<your_home>

I’m familiar with the idea of ‘Home’ on a unix box but in a Windows enviroment ? I wasn’t really sure what it meant (I’ve only been using Windows for 15 years so you can see where the confusion crept in). I managed to persuade myself it was referring to the Bellatrix directory within the Python site-packages directory !

Well here’s the news for the Windows users out there who are as ‘home-challenged’ as I am. It turns out you should putting those config files in a directory located at :

%HOMEPATH%\.bellatrix

which for me is

C:\Users\Richard Shea\.bellatrix

Yes but what about the dots ?

Which brings me onto another windows specific thing – how do you produce a directory with a dot as its first character ? I started off trying to do it via File Explorer and discovered something strange. File Explorer won’t let you specify a directory name that starts with a dot … unless you specify the directory name as having a trailing dot as well … if you do that File Explorer quietly throws away the trailing dot and you have the name you wanted in the first place ! Weird … or perhaps “only on Windows”

In closing

Anyway the good news is I’ve got it up and running and listing my instances ! Next step is to to use Bellatrix to start an EC2 instance and provision it as a Nagare environment. There’ll be a blog post about that by the time I’ve done, I’m sure !

Datejs – Date.isAfter – no such function ?

date.js – missing methods

Summary

Save yourself a lot of pain and ensure you’re using the right version of  Datejs.

Datejs – It’s great !

Let me start by saying that Datejs, an open-source JavaScript Date Library, is great ! It provides a lot of much needed functionality to the area of dates in Javascript.

Weirdly missing methods

Having said that what I want to highlight is that if you use the standard download links offered by the website (as shown below) you’ll get a version of Datejs which is not the most current one.

What’s more it seems that the version you will get contains a number of defects.

I can vouch for this as I’ve spent a couple of hours today wondering why the ‘.isAfter’ and the ‘.compare’ methods didn’t seem to exist in the form they are documented ! Very frustrating !

What You Should Do

Instead of using the download link on the first page to get the date.js file go here instead : http://www.datejs.com/build/date.js. The version number is the same, “1.0 Alpha-1″, but the build date is “2008-05-13″.

At least at the time of writing – it’s the “2008-05-13″ build you want.

Credit Where Credits Due

I’m grateful to Ben McIntyre whose post on the Datejs forums alerted me to this problem, I only wish I’d seen it earlier !

Find (Python) File in which object is defined

Every so often I have an object defined in some Python code and I want to work out in which file that object was defined.

Generally this can be done by a bit of grepping but if the object name in question is pretty generic this can be less than productive and it turns out there is a nice ready made way provided for us in the Python Standard Library.

So as an example currently I’m working with Pyro4 and I’m curious to know, for instance, where a particular constant, VERSION, is defined.

>>> import Pyro4
>>> print Pyro4.constants.VERSION
4.8

Well the ‘inspect’ module from the Python standard library allows you do just that, specifically the ‘getfile’ method – http://docs.python.org/library/inspect.html#inspect.getfile

>>> inspect.getfile(Pyro4.constants)
'C:\\Python27\\lib\\site-packages\\pyro4-4.8-py2.7.egg\\Pyro4\\constants.pyc'

As we can now see the VERSION object derives from the constants.pyc at the path given. In this case the definition is actually in the middle of an egg, pyro4-4.8-py2.7.egg,  which  means direct access to the underlying source code is not as straightforward as if it were implemented in plain old python scripts.

Nevertheless we can still make use of another inspect module function, the ‘getsource’ method (http://docs.python.org/library/inspect.html#inspect.getsource) to get some more information about the object we’re interested in as follows:

>>> inspect.getsource(Pyro4.constants)
'"""\nDefinitions of various hard coded constants.
Pyro - Python Remote Objects.  Copyright by Irmen de Jong.
irmen@razorvine.net - http://www.razorvine.net/projects/Pyro"""
# Pyro version\nVERSION = "4.8"
# standard object name for the Daemon object
DAEMON_NAME = "Pyro.Daemon"
# standard name for the Name server itself
NAMESERVER_NAME = "Pyro.NameServer"

… etc, etc.

The Python dict() constructor

The Python dict() constructor

Summary

How to add key/value pairs to an existing dictionary using the dict() contstructor

Using dict()

I’ve fallen into the habit of building dictionaries in Python using the braces approach, that is :

d1 = {'name':'Jane', 'age':21}

I was reminded today that you can use the conventional constructor method associated with the dict type as follows :

d1 = dict(name='Jane', age=21)

This will produce the same dictionary as the previous example. Notice that name of the keyword arguments (‘name’ and ‘age’) end up being the keys of the dictionary. Notice also that because they are keyword arguments to the function dict() they are not supplied as quoted strings.

What I learnt today

I was looking at some code today and discovered there’s something else the dict() function can do which I didn’t previously know of. If you have an existing dictionary which you wish to add some key/value pairs to you can do this.

#Create d1 from above
d1 = dict(name='Jane', age=21)

#Now produce a new dictionary, d2, based
#upon d1 and with extra key/value pairs

d2=dict(d1, weight=50, shoesize=7)

print d2
{'age': 21, 'shoesize': 7, 'name': 'Jane', 'weight': 50}

Taking it further

Not surprisingly you can use the same technique to modify the existing key/value pairs in a dictionary, like this :

#Create d1 from above
d1 = dict(name='Jane', age=21)

#Now produce a new dictionary, d3, based
#upon d1 with a modified existing key/value pair

d3=dict(d1, name='John')

print d3
{'age': 21, 'name': 'John'}

Where is Django installed ?

Where is Django installed ?

Summary

It’s useful to know where Django is for a number of reasons – customising admin templates for instance.

Todays Learning Point

Django is generally going to be installed in the site-packages directory of the Python directory you’re using. If you’re on an unfamiliar machine however where that is may not be immediately obvious so here’s a way to be sure.

If you need to know where the Django installation is you can do that from within Python quite easily.

>>> import django
>>> print django.__path__
['/usr/lib/python2.5/site-packages/django']

Why __path__

__path__ is a special attribute of Python packages; it’s initialized to hold the name of the directory holding the package’s __init__.py. To put that in blunter terms __path__ is going to tell you where the files that make up the package are – in this case Django.