Tag Archives: examples

Find (Python) File in which object is defined

Leave a reply

Every so often I have an object defined in some Python code and I want to work out in which file that object was defined.

Generally this can be done by a bit of grepping but if the object name in question is pretty generic this can be less than productive and it turns out there is a nice ready made way provided for us in the Python Standard Library.

So as an example currently I’m working with Pyro4 and I’m curious to know, for instance, where a particular constant, VERSION, is defined.

>>> import Pyro4
>>> print Pyro4.constants.VERSION
4.8

Well the ‘inspect’ module from the Python standard library allows you do just that, specifically the ‘getfile’ method – http://docs.python.org/library/inspect.html#inspect.getfile

>>> inspect.getfile(Pyro4.constants)
'C:\\Python27\\lib\\site-packages\\pyro4-4.8-py2.7.egg\\Pyro4\\constants.pyc'

As we can now see the VERSION object derives from the constants.pyc at the path given. In this case the definition is actually in the middle of an egg, pyro4-4.8-py2.7.egg,  which  means direct access to the underlying source code is not as straightforward as if it were implemented in plain old python scripts.

Nevertheless we can still make use of another inspect module function, the ‘getsource’ method (http://docs.python.org/library/inspect.html#inspect.getsource) to get some more information about the object we’re interested in as follows:

>>> inspect.getsource(Pyro4.constants)
'"""\nDefinitions of various hard coded constants.
Pyro - Python Remote Objects.  Copyright by Irmen de Jong.
irmen@razorvine.net - http://www.razorvine.net/projects/Pyro"""
# Pyro version\nVERSION = "4.8"
# standard object name for the Daemon object
DAEMON_NAME = "Pyro.Daemon"
# standard name for the Name server itself
NAMESERVER_NAME = "Pyro.NameServer"

… etc, etc.

The Python dict() constructor

Leave a reply

The Python dict() constructor

Summary

How to add key/value pairs to an existing dictionary using the dict() contstructor

Using dict()

I’ve fallen into the habit of building dictionaries in Python using the braces approach, that is :

d1 = {'name':'Jane', 'age':21}

I was reminded today that you can use the conventional constructor method associated with the dict type as follows :

d1 = dict(name='Jane', age=21)

This will produce the same dictionary as the previous example. Notice that name of the keyword arguments (‘name’ and ‘age’) end up being the keys of the dictionary. Notice also that because they are keyword arguments to the function dict() they are not supplied as quoted strings.

What I learnt today

I was looking at some code today and discovered there’s something else the dict() function can do which I didn’t previously know of. If you have an existing dictionary which you wish to add some key/value pairs to you can do this.

#Create d1 from above
d1 = dict(name='Jane', age=21)

#Now produce a new dictionary, d2, based
#upon d1 and with extra key/value pairs

d2=dict(d1, weight=50, shoesize=7)

print d2
{'age': 21, 'shoesize': 7, 'name': 'Jane', 'weight': 50}

Taking it further

Not surprisingly you can use the same technique to modify the existing key/value pairs in a dictionary, like this :

#Create d1 from above
d1 = dict(name='Jane', age=21)

#Now produce a new dictionary, d3, based
#upon d1 with a modified existing key/value pair

d3=dict(d1, name='John')

print d3
{'age': 21, 'name': 'John'}

What’s wrong with this ?

Leave a reply

What’s wrong with this ?

From a long list of things I’m amazed I didn’t see more quickly.

Summary

Do you sometimes find you go looking for a difficult problem when what you have is a simple problem ?

Todays Learning Point

Todays learning point is : if something weird is happening look at the simple things first.

 private bool _blnShowContractTabs;
 public bool blnShowContractTabs
 {
 get { return _blnShowContractTabs; }
 set { _blnShowContractTabs = value; }
 }

 private bool _blnShowAddEditLinks;
 public bool blnShowAddEditLinks
 {
 get { return _blnShowAddEditLinks; }
 set { _blnShowContractTabs = value; }
 }

I spent a little while today looking for a “big problem” which turned out to be a carelessly typed property set statement (… carelessly typed by me … I should add).

Zen ?

I was listening to the This Developers Life podcast the other day and Michael Moore spoke about problem solving and times when your problem isn’t really a problem at all (he expressed it much better than I could). I thought about Michael when I found this one !

Right on Date/Time

Leave a reply

Date/Time Named Files in DOS Batch

Producing date/time named files for output from DOS Batch

Summary

Good for when you want the output from a .BAT command file to be written to a date/time stamped file.

My Problem

Today I wanted to schedule a job to run on a clients machine several times a day for a number of weeks. Each time it ran I wanted an output file to with a name a bit like this :

DIAG-20100604T120000.txt

I find writing output to files which have their date/time of creation embedded in their names to be so useful it’s something I’m willing to muck around a bit to achieve.

The job i was running was all wrapped up in a .BAT file – how difficult could it be to get a .BAT file to produce a formatted date/time ? Well the answer is sufficiently difficult I side-stepped the problem !

Locale Independent

When I started looking for others who’d formatted date/times in .BAT land I found plenty of that made assumptions about the date/time format present on the machine it was running on but I really dislike using code like that – I’ve been bitten by it too many times in the past .

In the end I decided the only (or at least the easiest) thing to do was to allow a .VBS file to do the actual date/time formatting and allow the .BAT to use that.

First take some VBS

I pulled together a script I called iso.vbs.

The key thing with the VBS is the way the formatted date/time gets used in the last line. We’re going to use that output when we write some .BAT.

function addLeadingZeroIfNess(strIn)
 if len(strIn) < 2 then
 strOut = "0" & strIn
 else
 strOut = strIn
 end if

 addLeadingZeroIfNess = strOut

end function
'++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
OurDate = Now()
Yr = DatePart("yyyy",OurDate)
Mth = addLeadingZeroIfNess(DatePart("m",OurDate))
Dy = addLeadingZeroIfNess(DatePart("d",OurDate))
Hr = addLeadingZeroIfNess(DatePart("h",OurDate))
Mn = addLeadingZeroIfNess(DatePart("n",OurDate))
Sec = addLeadingZeroIfNess(DatePart("s",OurDate))
ISO = Yr & Mth & Dy & "T" & Hr & Mn & Sec
wscript.echo ISO

Now Add a bit of .BAT

Then I wrote a short .BAT file, iso.bat, which invokes the iso.vbs shown above and then invokes the ‘real’ .BAT which is the one I actually wanted to run, diag.bat

@echo off
for /f "delims=" %%a in ('cscript //nologo iso.vbs') do (set ISODateTime=%%a)
call diag.bat > "DIAG-"%ISODateTime%".txt"

iso.bat uses the formatted date/time  (eg “20100604T120000”) produced by iso.vbs above to assign to the variable %ISODateTime%. The variable is then available to form part of the file name used for the diag.bat output.

A Bit of Context

If the machine I was using had either PowerShell or Python installed Iwould have come at this from a different angle but … it didn’t !

Credit Where Credits Due

A posting by Tom Lavedas in the vistax64 forums put me on the right track for passing the values from VBS to BAT land – for which my thanks.

Taking your PIL with py2exe

Leave a reply

Letting PIL and py2exe be friends

Escape from “cannot identify image file” hell

Summary

The Python Imaging Library (PIL) provides great image manipulation help for your python programs. Py2exe provides a smart way of turning a python script into a windows executable. There are times however when the resulting .exe will break if you don’t do the right thing.

My Problem

I’d written a Python script, openpilwithdiag.py, which read TIFF files and worked just fine when run through the Python interpreter.

from PIL import Image
im = Image.open("test.TIFF")
print im.getcolors()

I then used py2exe to convert the script to a .exe.  Aargh – when I went to run it I got an error :

IOError: cannot identify image file

It took a little digging around but it seems that when PIL constructs one of its PIL Image objects it searches a set of ‘image plugin’ files within the PIL installation. This is all good except that by the time py2exe has done it’s thing PIL (now nicely wrapped up inside the .exe) will be looking in the wrong place.

The Solution

Pleasantly simple – given all the blind alleys I looked down to start with ! I’d previously had nasty problems using TIFF files with PIL and I think I started looking at this expecting the answer to be really nasty !

Whatever file formats you want to use import the corresponding PIL plugin directly into the script so that the instantiation of Image can ‘see’ them. For my example that’s like this :

from PIL import Image
from PIL import TiffImagePlugin
im = Image.open("test.TIFF")
print im.getcolors()

Tried py2exe again and what do we get ?

C:\>openpilwithdiag.exe
[(37306, (76, 25, 12, 51)), (42810, (0, 0, 0, 255)), (1489484, (0, 0, 0, 0))]

Much better !

Plugin Details

The set of possible plugins may be found by looking for files named xImagePlugin.py (where x is the name of the supported format) within the sub-directory of your Python installation at :

<your python directory>\Lib\site-packages\PIL

A list of file formats PIL supports is seen in the appendix of the PIL Handbook

Postscript

It occurs to me that for anyone who’s never touched py2exe it might be worth posting the setup.py I was using to drive the py2exe process – so here it is :

from distutils.core import setup
import glob
import py2exe

setup(console=['openpilwithdiag.py'],
            data_files=[]
)

Iterating over directories using .BAT

Leave a reply

Wildcard Directory Copy in DOS Batch

Moving lots of directories using DOS Batch and wild cards

Summary

This is great for those hideous clean up jobs where you have masses of sub-directories and you want to move a large subset of them somewhere else.

My Problem

I had a directory and it was full of sub-directories looking like this :


2009-12-01 07:33 111-y-888
2009-12-01 07:32 123-x-456
2009-12-01 07:32 123-y-456
2009-12-01 07:32 125-y-456
2009-12-01 07:33 876-x-342
2009-12-01 07:33 999-x-123
...

There were thousands of these things and I only wanted to remove the ones that had an ‘x’ in the middle.

No problem I thought XCopy will do this. Well strange to say but if you want to use wild cards – and clearly I did – XCopy won’t drill down and copy the contents of the director you’re copying. You can use the /E flag and get an empty directory copied but not the actual contents.

Weirdo DOS Batch arguments !

Weirdo DOS Batch arguments to the rescue ! I wrote a one line .BAT file like this :

for /d %%X in (%1) do move %%X %2\%%~nX

I called that movedirs.bat and then to move all the ‘x’ directories from E:\ABC to E:\XYZ I called it like this

movedirs.bat E:\ABC\*x* E:\XYZ

which uses a for loop to process each directory which matches the wildcard individually and uses xcopy on each sub-directory within the for loop.

The gory details

The /d argument on the ‘for’ ensures that only directories are processed. The ‘~n’ modifier on the %%X variable means that the original sub-directory name (as opposed to the entire path) is used as the target within the second command line argument.

So good I wrote it twice

Just to be up front this post is based on a stackoverflow question I posed and then answered myself. It seemed too useful to just be there so I recycled it for my blog.

Hex to Decimal Conversion in .NET

Leave a reply

Hex to Decimal in C#

Two alternate ways of converting a hex string to decimal

Summary

The other day I needed to write some code to convert an integer expressed in hexadecimal to decimal. I was surprised to find that the way to do it wasn’t the way I expected.

The value I was processing had come from a different environment and had a ‘0x’ prefix to define its base – it was also a string. I just assumed i could feed this to int.Parse and I’d get the answer I was after.

In fact it seems that you either have to get rid of the ‘0x’ prefix or find some other way of doing it. I write a little program to illustrate the choices.

AllowHexSpecifier Weirdness

One thing to notice here which is really quite strange is that the int.Parse option takes an enumerated type argument System.Globalization.NumberStyles.AllowHexSpecifier when in fact it doesn’t allow a Hex specifier ! I’m sure there’s some sense to that but I don’t see it myself !

Example Code

using System;
using System.Collections.Generic;
using System.Text;
using System.Globalization;

namespace HexParsing
{
    class Program
    {
        static void Main()
        {
            string strTestHex ;
            int iTestHex;
            string sOut;
            string s1 = "{0} converted to base 10 using int.Parse is {1}";
            string s2 = "{0} converted to base 10 using Convert.ToInt32 is {1}";

            //Without a prefix - using int.Parse
            strTestHex = "7FC3DAE0";
            iTestHex = int.Parse(   strTestHex,
                                    System.Globalization.NumberStyles.AllowHexSpecifier,
                                    CultureInfo.CurrentCulture);
            sOut = String.Format(CultureInfo.CurrentCulture, s1, strTestHex, iTestHex);
            Console.WriteLine(sOut);

            //With a prefix - using Convert.ToInt32
            strTestHex = "0x7FC3DAE0";
            iTestHex = Convert.ToInt32(strTestHex,16);
            sOut = String.Format(CultureInfo.CurrentCulture, s2, strTestHex, iTestHex);
            Console.WriteLine(sOut);

            Console.WriteLine("Press ENTER to close window");
            Console.ReadLine();

        }
    }
}

Sphinx Documentation Generator – Tutorial 2

Leave a reply

Sphinx – first project output

A series of tutorials on using the Sphinx Documentation system aimed at absolute beginners.

Summary

In this second part of my Sphinx Tutorial we use the ‘quickstart’ script provided with Sphinx to quickly produce a basic set of documentation source files.

Our First Sphinx Project

Sphinx has a great way to get you started and that’s the ‘quickstart’ script (you’ll find sphinx-quickstart-script.py in the Scripts directory of your Python installation). Quickstart walks you through a series of questions to produce a basic set of the files you’ll need to produce your documentation.

Quickstart is pretty easy to use and even more so when your aim is to produce something very basic (which is the aim of this tutorial).

We’ll just step through the more important bits here :

Specify where you want the Documentation to be created

Sphinx is going to create some files and a few directories so choose where you want those to be created :
Enter the root path for documentation.
> Root path for the documentation [.]: C:\data\src\Sphinx\Test1

Specify a name for the Documentation:

What ever name you choose will appear in various places in the documentation so choose something you’re happy with

The project name will occur in several places in the built documentation.
> Project name: Sphinx Tutorial
> Author name(s): Richard Shea

Specify a version for the Documentation:

Similarly you’ll get the version number appearing in various parts of the output documenation

> Project version: 1.0.0

Specify if you want any clever stuff:

Sphinx is able to do all sorts of clever stuff apart from interpreting your RST files into beautiful output. We don’t want any of that for the first part of the tutorial so say ‘no’ to all of this.

Please indicate if you want to use one of the following Sphinx extensions:
> autodoc: automatically insert docstrings from modules (y/N) [n]:
> doctest: automatically test code snippets in doctest blocks (y/N) [n]:
> intersphinx: link between Sphinx documentation of different projects (y/N) [n]:
> todo: write "todo" entries that can be shown or hidden on build (y/N) [n]:
> coverage: checks for documentation coverage (y/N) [n]:
> pngmath: include math, rendered as PNG images (y/N) [n]:
> jsmath: include math, rendered in the browser by JSMath (y/N) [n]:
> ifconfig: conditional inclusion of content based on config values (y/N) [n]:

Make it easy on yourself (at least to start with)

It’s perfectly reasonable to use build the Sphinx output by providing your own command line options but to get started with it’s helpful to allow Sphinx to help a little so say ‘yes’ to a .BAT or a makefile (depending on what’s your choice of poison).

A Makefile and a Windows command file can be generated for you so that you
only have to run e.g. `make html' instead of invoking sphinx-build
directly.
> Create Makefile? (Y/n) [y]:
> Create Windows command file? (Y/n) [y]:

The results of quickstart

So after all that what have we got ? Well we specified C:\data\src\Sphinx\Test1 as the output path and if we look in there we find we’ve got a build and a source directory (as well as the Makefile and make.bat we asked for). What we’re interested in to start with is source\index.rst.

Open it up in a text editor and you’ll see it looks like this :
.. Sphinx Tutorial documentation master file, created by
sphinx-quickstart on Sun Nov 1 18:08:19 2009.
You can adapt this file completely to your liking, but it should at least
contain the root `toctree` directive.

Welcome to Sphinx Tutorial's documentation!
===========================================

Contents:

.. toctree::
:maxdepth: 2

Indices and tables
==================

* :ref:`genindex`
* :ref:`modindex`
* :ref:`search`

Next Steps

In the next part of this series, Sphinx – Absolute Beginners – Part 3, we’ll see how to make this file into either HTML or PDF output. In later parts of the series we’ll see how to use your existing Python code comments to produce automated documentation of your Python code.

Sphinx Documentation Generator – Tutorial 1

Leave a reply

Sphinx – Absolute Beginners – Part 1

A series of tutorials on using the Sphinx Documentation system aimed at absolute beginners

Summary

Sphinx is a great tool that makes it easy to create good looking documentation in a range of output formats. If you’re a Python programmer it also makes it easy to use your existing docstring comments to form the basis of your documentation.

I’ve found using Sphinx for the first time a little difficult as much of the documentation seems to assume you know certain things. This series of posts provides a tutorial for the absolute beginner.

Ground Rules (for impatient programmers)

Now at this stage there’s a couple of things that are well worth knowing !

  • Sphinx is good at auto-documenting Python code but that’s not its primary role. If you’re a Python programmer looking to auto-document your code it takes a while to understand that a lot of the facilities in Sphinx are nothing to do with what you’re after and that can be confusing.
  • In Sphinx-land ‘the source files’ refer to the documentation configuration files. This confused me for a while whenever the term ‘source files’ was used I thought it meant my python source files.

Getting Spinx Installed

The prerequisites part of the Sphinx doco does a good job here but in summary:

  • You want to have installed Python 2.4 or better
  • Grab the source from the Sphinx Python Project Page or …
  • … just use easy_install -U Sphinx

Next Steps

Sphinx has a great way to get you started and that’s the ‘quickstart’ script. In the next part of this tutorial series we’ll cover how to quickly and easily produce your first Sphinx project.