Find (Python) File in which object is defined

Every so often I have an object defined in some Python code and I want to work out in which file that object was defined.

Generally this can be done by a bit of grepping but if the object name in question is pretty generic this can be less than productive and it turns out there is a nice ready made way provided for us in the Python Standard Library.

So as an example currently I’m working with Pyro4 and I’m curious to know, for instance, where a particular constant, VERSION, is defined.

>>> import Pyro4
>>> print Pyro4.constants.VERSION
4.8

Well the ‘inspect’ module from the Python standard library allows you do just that, specifically the ‘getfile’ method – http://docs.python.org/library/inspect.html#inspect.getfile

>>> inspect.getfile(Pyro4.constants)
'C:\\Python27\\lib\\site-packages\\pyro4-4.8-py2.7.egg\\Pyro4\\constants.pyc'

As we can now see the VERSION object derives from the constants.pyc at the path given. In this case the definition is actually in the middle of an egg, pyro4-4.8-py2.7.egg,  which  means direct access to the underlying source code is not as straightforward as if it were implemented in plain old python scripts.

Nevertheless we can still make use of another inspect module function, the ‘getsource’ method (http://docs.python.org/library/inspect.html#inspect.getsource) to get some more information about the object we’re interested in as follows:

>>> inspect.getsource(Pyro4.constants)
'"""\nDefinitions of various hard coded constants.
Pyro - Python Remote Objects.  Copyright by Irmen de Jong.
irmen@razorvine.net - http://www.razorvine.net/projects/Pyro"""
# Pyro version\nVERSION = "4.8"
# standard object name for the Daemon object
DAEMON_NAME = "Pyro.Daemon"
# standard name for the Name server itself
NAMESERVER_NAME = "Pyro.NameServer"

… etc, etc.

The Python dict() constructor

The Python dict() constructor

Summary

How to add key/value pairs to an existing dictionary using the dict() contstructor

Using dict()

I’ve fallen into the habit of building dictionaries in Python using the braces approach, that is :

d1 = {'name':'Jane', 'age':21}

I was reminded today that you can use the conventional constructor method associated with the dict type as follows :

d1 = dict(name='Jane', age=21)

This will produce the same dictionary as the previous example. Notice that name of the keyword arguments (‘name’ and ‘age’) end up being the keys of the dictionary. Notice also that because they are keyword arguments to the function dict() they are not supplied as quoted strings.

What I learnt today

I was looking at some code today and discovered there’s something else the dict() function can do which I didn’t previously know of. If you have an existing dictionary which you wish to add some key/value pairs to you can do this.

#Create d1 from above
d1 = dict(name='Jane', age=21)

#Now produce a new dictionary, d2, based
#upon d1 and with extra key/value pairs

d2=dict(d1, weight=50, shoesize=7)

print d2
{'age': 21, 'shoesize': 7, 'name': 'Jane', 'weight': 50}

Taking it further

Not surprisingly you can use the same technique to modify the existing key/value pairs in a dictionary, like this :

#Create d1 from above
d1 = dict(name='Jane', age=21)

#Now produce a new dictionary, d3, based
#upon d1 with a modified existing key/value pair

d3=dict(d1, name='John')

print d3
{'age': 21, 'name': 'John'}

Where is Django installed ?

Where is Django installed ?

Summary

It’s useful to know where Django is for a number of reasons – customising admin templates for instance.

Todays Learning Point

Django is generally going to be installed in the site-packages directory of the Python directory you’re using. If you’re on an unfamiliar machine however where that is may not be immediately obvious so here’s a way to be sure.

If you need to know where the Django installation is you can do that from within Python quite easily.

>>> import django
>>> print django.__path__
['/usr/lib/python2.5/site-packages/django']

Why __path__

__path__ is a special attribute of Python packages; it’s initialized to hold the name of the directory holding the package’s __init__.py. To put that in blunter terms __path__ is going to tell you where the files that make up the package are – in this case Django.

Django to the world

Django to the world

When your shiny new Django site is invisible to other machines …

Summary

The default setting in Django means your development server is invisible to other machines

Todays Learning Point

When a developer creates a nice new Django site and uses the django-admin.py script :

django-admin.py runserver

to start the development server. By default the development server is responding to requests made on port 8000 on IP address 127.0.0.1 (or the synonym ‘localhost’). As such you’re not going to be able to see that Django site from any other machine.

In most cases this is just what’s needed. The development server is intended for use by the developer only. However there may be circumstances where you want another developer to see your work – or as happened to me today where you are developing on a virtual machine running within your development machine.

If that’s the case there’s a way around it

Specify IP on Server Launch

You can issue a slightly different command when starting the development server

python manage.py runserver 0.0.0.0:8000

The above command will listen on port 8000 of all public IP addresses of the hosting machine and that in turn will mean other machines can access the Django site served through your development server.

Ace ‘Editor in the Browser’ – Documentation

Ace ‘Editor in the Browser’ – Documentation

The documentation for Ace – where is it ?

Summary

The Ace documentation isn’t that hard to find but you do need to know where to look

Ace in the hole

The Ace project provides a customizable, embeddable code editor for web applications. It’s written solely in JavaScript and is really very impressive.

You can see it in action at their demo page (which frankly is a bit bare bones) or at the Cloud9 project for which Ace is the starting point.

Documentation is where ?

You can download the Ace distribution from https://github.com/ajaxorg/ace – you need the ‘Downloads’ button at the top right.

Once you’ve done that you’ll find there’s nothing worth reading in the docs folder within the distribution.

Instead what you need to do is go visit the Ace wiki at Github. It took me a while to work this out (with the help of the Ace user group forum) so I hope it will save some others the trouble.

AttributeError: ‘str’ object has no attribute ‘digits’

‘str’ object has no attribute ‘digits’

On silly ways you can puzzle yourself – part 412

Summary

How to make Python report that a string object has no attribute ‘digits’

Confusing Yourself – the easy way


Today I was working on a little piece of code which I hadn’t originally written and which looked something like this :

import string
def foo(string):
  for c in string:
    if c in string.digits:
       #do something

As is well known the Python string module contains a number of useful constants one of which is string.digits

>>> import string
>>> print string.digits
0123456789

Missing ‘digits’

My problem was that every time I went to execute this code it got to the reference to string.digits and the Python intepreter would report

AttributeError: 'str' object has no attribute 'digits'

I spent a happy half hour looking backwards and forwards trying to understand why the String module might think it had no attribute ‘digits’ when everything indicated quite clearly it did until I realised what the problem was.

def foo(string):

That argument name ‘string’ had carefully chucked away my previous reference to the String module and as a string has no attribute ‘digits’ the intpreter was quite reasonably complaining !

My defense

In my defense I wouldn’t normally use variable names, like ‘string’,  that come quite that close to commonly used modules but then like I say I didn’t write the original function

… but what I should have done

But then again what I should have done a great deal sooner than I did was to add a couple of lines to the function so that it read :

import string
import pprint
def foo(string):
  for c in string:
    pprint.pprint(dir(string))
    if c in string.digits:
       #do something

which would have output something like this

['__add__',
 '__class__',
 '__contains__',
 '__delattr__',
 '__doc__',
 '__eq__',
 '__format__',
 '__ge__',
 '__getattribute__',
 '__getitem__',
 '__getnewargs__',
 '__getslice__',
 '__gt__',
 '__hash__',
 '__init__',
 '__le__',
 '__len__',
 '__lt__',
 '__mod__',
 '__mul__',
 '__ne__',
 '__new__',
 '__reduce__',
 '__reduce_ex__',
 '__repr__',
 '__rmod__',
 '__rmul__',
 '__setattr__',
 '__sizeof__',
 '__str__',
 '__subclasshook__',
 '_formatter_field_name_split',
 '_formatter_parser',
 'capitalize',
 'center',
 'count',
 'decode',
 'encode',
 'endswith',
 'expandtabs',
 'find',
 'format',
 'index',
 'isalnum',
 'isalpha',
 'isdigit',
 'islower',
 'isspace',
 'istitle',
 'isupper',
 'join',
 'ljust',
 'lower',
 'lstrip',
 'partition',
 'replace',
 'rfind',
 'rindex',
 'rjust',
 'rpartition',
 'rsplit',
 'rstrip',
 'split',
 'splitlines',
 'startswith',
 'strip',
 'swapcase',
 'title',
 'translate',
 'upper',
 'zfill']

… and made it pretty clear that things were not as I thought they were.

Sphinx – how to make autodoc really automatic ?

Automating sphinx.ext.autodoc

A tool to help use the ‘autodoc’ facilties of Sphinx.

Summary

Sphinx has a great extension,sphinx.ext.autodoc, which imports the modules you want to document and uses the docstrings as the basis for the documentation. A script I’ve just found makes using sphinx.ext.autodoc even easier.

generate_modules.py

Although the sphinx.ext.autodoc extension reduces the work of creating Sphinx source files a great deal I still found myself having to create a set of files which corresponded to the modules in a project. This was a bit of a bore particularly as I like to start creating documentation early in a project and so modules would come and go during development.

On a number of occasions I wished I had a script to automatically create the source files needed … well it turns out that Etienne Desautels has already done the heavy lifting and written generate_modules.py which does just what I want.

Making use of it

generate_modules.py is completely independent of Sphinx. Just download it from the above location and run it as

>python generate_modules.py --help

Usage: generate_modules.py [options] <package path> [exclude paths, ...]

Note: By default this script will not overwrite already created files.

Options:
 -h, --help            show this help message and exit
 -n HEADER, --doc-header=HEADER
                       Documentation Header (default=Project)
 -d DESTDIR, --dest-dir=DESTDIR
                       Output destination directory
 -s SUFFIX, --suffix=SUFFIX
                       module suffix (default=txt)
 -m MAXDEPTH, --maxdepth=MAXDEPTH
                       Maximum depth of submodules to show in the TOC
                       (default=4)
 -r, --dry-run         Run the script without creating the files
 -f, --force           Overwrite all the files
 -t, --no-toc          Don't create the table of content file

Most of this is pretty self-explanatory.

My index.rst looks like this :

Welcome to pySourceAid's documentation!
=======================================

Contents:
=========

.. toctree::
:maxdepth: 2

overview.rst

Modules
===============

.. toctree::
:maxdepth: 20
:numbered:
:glob:

modules/*

Indices and tables
==================

* :ref:`genindex`
* :ref:`modindex`
* :ref:`search`

And that line in the index.rst

    modules/*

… means that autodoc is going to go looking for source files corresponding to each of the modules in a directory called modules. As a result when I run generate_modules.py I use a command like this

python generate_modules.py --suffix=rst --dest-dir=C:\myproject\docs\source\modules C:\myproject\src

Where the modules are living in C:\myproject\src

Environment

This blog post is based on some work done using : Python 2.6 and Sphinx 1.0.5

Sandcastle XML Reference

Sandcastle XML Reference

Handy reference to Sandcastle XML

Summary

Hard working Michael Sorens has produced a wall chart documenting the XML used with the Sandcastle Document Generator.

Reference to Sandcastle’s XML Documentation Comments

In a previous post about Sandcastle I mentioned a useful guide to the Sandcastle Documentation Generator written by Micheal Sorens.

I’ve just come across an accompanying wall chart.

Sandcastle XML Reference Sample

Sandcastle XML Reference Sample

The wall chart provides a quick reference to the XML used by a programmer in source code comments to allow Sandcastle to automatically generate documentation.

The wall chart is a good resource for anyone using Sandcastle and is available for download as a PDF here.

EDIT: The link to the wallchart went bad after this post was published. I’ve now amended the link to point to a version of the wallchart which works as at April 2012.

Sandcastle – the whole story !

Sandcastle – the whole story !

Documenting a document generator !

Summary

Sandcastle Document Generator is a great tool for producing project documentation but getting it to work can be … challenging ! Here’s some resources to help.

How does this thing work ?

If you’ve used Sandcastle and XML Documentation to automatically produce your .NET project documentation you’ll know:

  • the resulting output is great
  • getting the output can be challenging !

I came across an article by Michael Sorens which very comprehensively documents the process you need to go through.

Taming Sandcastle: A .NET Programmer’s Guide to Documenting Your Code is (as the intro says) “the easy guide to the process that Microsoft never managed, and introduces several applications that help” – it’s so much better than the process I went through when I was first using Sandcastle – I highly recommend it.

XML Documentation ? – How’s that ?

Now backtracking a little. For those who are not fully on the XML Documentation train here are some other good resources:

  • GhostDoc – Great Visual Studio Extension for ‘automagically’ producing XML Documentation of classes; methods; and properties – it’s very good indeed and … it’s free !
  • Within Michaels main article there’s a nice section introducing the benefits of using XML Documentation
  • Having read the intro mentioned above the nice people at Dynicity have produced a very comprehensive referenceto XML Documentation Comments of all the options available.

What’s wrong with this ?

What’s wrong with this ?

From a long list of things I’m amazed I didn’t see more quickly.

Summary

Do you sometimes find you go looking for a difficult problem when what you have is a simple problem ?

Todays Learning Point

Todays learning point is : if something weird is happening look at the simple things first.

 private bool _blnShowContractTabs;
 public bool blnShowContractTabs
 {
 get { return _blnShowContractTabs; }
 set { _blnShowContractTabs = value; }
 }

 private bool _blnShowAddEditLinks;
 public bool blnShowAddEditLinks
 {
 get { return _blnShowAddEditLinks; }
 set { _blnShowContractTabs = value; }
 }

I spent a little while today looking for a “big problem” which turned out to be a carelessly typed property set statement (… carelessly typed by me … I should add).

Zen ?

I was listening to the This Developers Life podcast the other day and Michael Moore spoke about problem solving and times when your problem isn’t really a problem at all (he expressed it much better than I could). I thought about Michael when I found this one !