Mobicules

My long-time friends Niraj and Munish have started up a new company Mobicules. The company works in the areas of Web 2.0, social networking applications and related areas. They have been in operation for about a year now and have been doing well.
Both of them worked for Mentor (EDA) for about 5 years.
The company website is http://www.mobicules.com/.

Zope 3 Views as multi-adapters

This diagram shows how Zope Views are implemented as multi-adapters. Student is a content class which implements IStudent interface. HTTPRequest objects are created when an HTTP request (either through Web browsers, or through XML-RPC calls) is made. StudentView is an adapter which adapts a Student object along with an HTTPRequest object and implements IBrowserView interface.

The diagram is not accurate reflection of design of browser views in Zope. It is just for demonstration of the idea of multi-adapters.

Cron in Python

Cron jobs are traditional time based scheduling service on Unix platforms. For an introduction to cron jobs, go to: http://en.wikipedia.org/wiki/Crontab

At times, different software solutions may need to incorporate cron like functionality. I ended up doing the same recently. As usual, I first tried to find out if there are any existing implementations of cron available so that I do not have to re-invent the wheel again. I was looking for a Python based implementation and found the following: http://sourceforge.net/projects/pycron/ (Python Cron, Great Cron for Windows).

Although the title mentions Windows but the implementation can work on any Python based environment. The software is GPL licensed. Its a small and simple implementation of Cron in Python. The solution is easy to read and understand. It helped me understand some of the intricacies of cron implementation (like matching the cron table information with current time, taking care of system clock issues, etc.).

The algorithm is based on the initial simple implementation of cron by Brian Kernighan.

1. Read the cron file
2. Determine if any commands are to be run at the current date and time and if so, run them.
3. Sleep for one minute
4. Repeat from step 1.

Unicode Email in Python

Unicode email in Python requires quite good amount of care.

Read http://mg.pov.lt/blog/unicode-emails-in-python to know about it more.

SVN configuration file

Lot of people like me don't know about a configuration file in subversion on the client side which can help in setting up a lot of default options for them. It took me a time to figure it out, so I guess its worth talking about.

The file resides at: %APPDATA%\Subversion\config on Windows systems.

Some of the uses of this file are:

• Identifying different file extensions which should be ignored by default
• Identifying different file extensions for which svn:eol-style=native can be set by default.

Working with the SubVersion Config file

In the following I will show you some examples of how you can use this config file to make your life easier.

Global Ignores

Typically when I work on my python projects, a number of *.pyc files are generated when Python compiles my source files. It is annoying to see them listed whenever I run svn update or svn status commands. Fortunately one can specify a set of file name patterns (typically wild-card based) which will be ignored by default.

In your configuration file:

• Search for a line with keyword global-ignores
• If the line is commented, un-comment it
• add a list of file patterns which you wish to ignore

On my side it looks like:

global-ignores = *.o *.lo *.la *.pyc *.obj .DS_Store

VC++ programmers may like to add *.obj, *.exe, *.lib, *.dll etc.

Auto Props

SVN allows you to set some properties on the files. Some of the commonly used ones are svn:externals, svn:eol-style, svn:keywords etc.

I work on multiple platforms (Windows/Mac/Linux). Hence I usually need to check-in my source code with svn:eol-style=native so that when it is checked out on different platforms, the end of line characters are adjusted accordingly. (Read more about it in SubVersion manual).

My problem was that I had to do this manually for each file. Later on I found that its possible to specify in the configuration file a few automatic properties which get attached to a new file the moment its added to SVN repository.

• Find out a line which has enable-auto-props
• Make sure its un-commented
• Make sure that it looks like: enable-auto-props=yes
• Find out a section [auto-props]
• For each file pattern of your interest (*.c, *.cpp, *.py), define the automatic properties which you would like to be setup. e.g.: *.py = svn:eol-style=native;svn:keywords=Id
• You are done!!!

SubClipse and SVN

SubClipse supports two different SVN interfaces. JavaHL and SVNKit. JavaHL uses the same configuration files as the command-line client viz. %APPDATA%\Subversion\. SVNKit caches information in Eclipse keyring. I don't know much about it and won't talk about it. I am happy with JavaHL SVN interface for SubClipse.

I hope this post is useful to SubVersion users around the world.

AJAX and Browser Caching Problem

For a long time, I have not been able to understand how to bypass the browser caching problem for developing AJAX applications. Internet Explorer simply looks at the URL, sees whether it has it in the CACHE, and brings up the cached version whenever possible.



The caching problem is observed for HTTP GET requests only. There is no caching issue for HTTP POST requests. It turns out that this is actually very easy to achieve.

A URL is different if it has any of the GET request keys are different. Hence to bypass caching, all one needs to do is add a parameter to the URL which has a new value everytime the URL is accessed, and is not really used by the client/server programs.

For more information see:
http://www.spacebug.com/solving_browser_caching_problem_of_ajax.html

http://www.howtoadvice.com/StopCaching

Running Python Scripts with Administrator Privileges on MAC OS X

Problem

I had a piece of software written in Python. I come primarily from Windows background where permissions etc. are quite hazy. 

On MAC OS X I wanted this software to be able to run with Administrator privileges. Apple has an Authentication API which enables an application to request for administrator privileges from the user whenever it needs it. Unfortunately I couldn't find out any way to access the authentication API directly from Python code.

Solution

AppleScript (MAC's built-in scripting language) has a very simple and straightfoward way to execute any script with admin privileges. 

Lets look at a simple example first:

do shell script ¬
 "echo 'abc' > /Applications/MyTestFile" with administrator privileges

You can open Script Editor from /Applications/AppleScript/Script Editor.app and write the above example.

Interesting thing to note here is:

• I am trying to do something which will not succeed with administrator privileges
• The "with administrator privileges" clause ensures that AppleScript will ask you to grant administrator privileges to the shell script before execution
• If you don't grant administrator privileges (may be you don't know or you forgot the password, or may be you don't want the shell script to run), the script will not be run.
• If you grant the privileges, it will do as programmed above; i.e. write 'abc' to a file in /Applications.

Administrator privileges for Python programs

It must be easy for you to guess how to request for administrator privileges for running a Python program. Let me just put the code for clarity:

do shell script ¬
 "/usr/bin/python myPythonProgram.py" with administrator privileges

Limitations

• The solution requires you to request for privileges even before the python program started running. Thus if you wanted something where you could request the privileges only on need basis, this solution won't work. But at least something is better than nothing.

SVK for SVN users

This post is about setting up SVK for SVN users. I assume you know enough about SVN, and you are looking for a quick documentation which can help in getting off the mark with SVK. 

Objectives

At the end of reading this article you should:

• Understand how to mirror an SVN repository
• Understand how to Synchronize development between your local SVK repository and your central SVN repository
• Understand how to create local development branches
• Know typical day to day tricks for SVK usage

Introduction

One problem with SVN is that you cannot take advantage of it when you are offline. Thats where SVK comes in which provides you the capability of distributed version control. 

Using SVK you:

• Mirror an SVN repository in your local file system
• Do your day to day development using the local SVK repository (online/offline)
• Publish your changes back to the central SVN repository when deemed suitable (and when you are online)

Installation

Installation for different platforms is fairly well covered in

 http://svk.bestpractical.com/view/InstallingSVK

I just would like to let you know that I installed on my MAC and it was straightforward, without any surprises, and worked out of the box. 

SVK depotmap

SVK depot is a local database that SVK creates in which the mirrors of your SVN repositories are maintained .

Creating SVK depot

$svk depotmap --init

It will create a .svk directory inside your home directory. This is your default depot.

Setting up a mirror

Suppose you have the following svn repository structure:

svn://192.199.4.232/repos
- third-party
- mycalc
  - trunk
    - gui
    - arithmetic
  - tags
  - branches
- crm
  - trunk
  - tags
  - branches

Now you want to mirror the whole of trunk of your mycalc project. 

$svk mirror svn://192.199.4.232/repos/mycalc/trunk  //mycalc/trunk

The above will mirror the svn repository into your default svk depot

A bit of explanation

The command is:

svk mirror url_of_svn_repository path_to_local_mirror

The path to local mirror is formed as:

/depot_name/your_chosen_path

• The default depot has no name. Hence // above
• I chose to use mycalc/trunk as the local mirror path. This way I would essentially replicate the same directory structure as is there in the central repository, avoiding other parts of the structure which I don't wish to replicate (like branches and tags)

To see the list of mirrors, please try:

$svk mirror --list

Path            Source

=====================================================

//mycalc/trunk   svn://192.199.4.232/repos/mycalc/trunk

Note that the mirror doesn't yet have anything

Synchronizing the mirror with repository

$svk sync //mycalc/trunk

This will bring your local mirror //mycalc/trunk in sync with the central repository: svn://192.199.4.232/repos/mycalc/trunk. 

• Entire revision history for this repository is downloaded from central SVN server
• This operation would consume quite a lot of time depending upon how large the repository is

Note: This brings in the entire history of a repository to local repository. So it is very time consuming. It may not be what you want. See below on import.

Working with your local repository

The SVK commands are pretty much the same as SVN commands:

• To checkout : svk co //mycalc/trunk
• To update : svk update
• To commit : svk commit
• To see the status: svk status
• Listing the contents of a folder on repository: svk ls //mycalc/trunk/gui

Receiving other changes from central repository

Other people are working also and committing their changes to central repository. To receive those changes into your repository, try:

$svk pull //mycalc/trunk

Publishing your changes to central repository

When you are ready to publish your changes to central repository, try: 

$svn push //mycalc/trunk

SVK Star Merge

Subversion as of now doesn't support automatic merges. One has to record the revision numbers before merging code between two branches. SVK implements Star Merge algorithm first introduced in GNU Arch which makes merging code between branches a lot easier. 

More on this later..

SVK Integration with GUI clients

I haven't found any integration with GUI clients yet. May be sometime in future! Who knows!

My WishList

SVK + Subclipse : Most of my development so far has been done using Eclipse with the SubClipse plugin for SVN integration. I would very much love to have SVK support built-into SubClipse itself.

Python Socket Library Hostname quirks

Python socket library has some pretty useful functions and probably a weird behavior?:

Objective
I wanted to setup a portable way to detect whether I am connected to network or not.

When a computer is connected to some TCP/IP network (using either static or dynamic IP), it gets an IP address. Otherwise by default it has got a loopback IP address which is 127.0.0.1.

So I thought that I could use this idea for detection of presence of network.


Problem


>>> import socket # imports the socket library
>>> print socket.gethostname()
shailesh-macbook-pro.local


The above is on the my macbook-pro laptop.

If I do the same on my windows laptop I get:
>>> import socket # imports the socket library
>>> print socket.gethostname()
shailesh_lap


So far so good.

Things get weird with the following:

On Windows
>>> print socket.gethostbyname(socket.gethostname())
192.9.199.152

And on MAC Book Pro

>>> print socket.gethostbyname(socket.gethostname())
...
socket.gaierror: (8, 'nodename nor servname provided, or not known')

Now I don't really know what went wrong here. I had just sometime back restarted the system.
After some time, I found that this started working fine again

On MAC Book Pro Again after some time
>>> print socket.gethostbyname(socket.gethostname())
192.9.199.159

It is my guess that either Python on MAC, or MAC OS itself takes some time in setting up some of these IP address hostname mappings for the local host.

Work Around

But I needed to get the IP address of the host computer. My objective was to identify whether my computer is connected to some network or not. So I eventually sat down and wrote the following:

def gethostaddress(): 
    hostname = socket.gethostname() 
    hostaddr = u'' 
    try: 
        hostaddr = socket.gethostbyaddr(hostname) 
    except socket.gaierror, e: 
        try: 
            hostaddr = socket.gethostbyaddr(u'localhost') 
        except: pass 
    except: pass 
    return hostaddr 

def networkIsPresent(): 
    return u'127.0.0.1' == gethostaddress()

 

Although this may not be perfect, but I guess I will work with this for now, till I have a better solution.

Background of this blog

Python is one of my favorite programming languages. (The other ones C/C++).

I have done a lot of Python based development over the last two years and learnt quite a few tricks.

Have always thought of writing a book on Python, but that seems to be too grand a project, and I probably don't yet know enough to be able to do justice to such an attempt.

Nevertheless I can at least start blogging on Python.

Hence this blog.

UPDATE

During development, I usually use a lot of other tools too. I wanted to write about them also. Initially I thought to create a separate blog for general software related activities. But later I decided, it won't be possible for me to maintain multiple blogs. Hence I am renaming this blog to Shailesh on Software Development and will collect all my discussions related to software under this blog only.