Michael Thornhill: September 2005

This post gives a quick introduction on how to debug Filesystem Products in Zope/Plone with the following IDEs

Wing IDE Professional 2.0.3
Boa Constructor 0.4.4
Eclipse 3.1 with PyDev 0.98
Emacs 21.4.1 with pdb and ipython 0.6.15
jEdit 4.2final with JpyDbg 0.9.1

If anyone knows of any other suitable IDEs (or if I've left out features in the ones above), please feel free to let me know in the comment section at the bottom of this post. This post doesn't aim to say which IDE is better than another, just to show how to use them for debugging.
This post is a follow on from Team Development with Plone/ Zope/ ZEO/ Subversion/ ipython. You will need to run these commands to try out the IDE examples in this post.

mkdir ~/plonedev
cd ~/plonedev
wget -c http://homepage.eircom.net/~rachra/setupplonezeo.sh
wget -c http://homepage.eircom.net/~rachra/plonehelper.py
chmod +x setupplonezeo.sh
./setupplonezeo.sh 1 Acme

Running the setupplonezeo.sh script will leave you with the directory structure shown below

~/plonedev/    
    startPlone2.1-rc3.sh        # this script starts the zeo server on port 8100 and plone listening on port 8080
    startIPythonPlone2.1-rc3.sh # this script starts an IPython interpreter on your zeo_client1 (port 9080)
    debugPlone2.1-rc3.py        # a python script to start zeo_client0 (port 8080)
    plonehelper.py              # python helper script for creating a plone instance
                                # installing products and selecting a skin using the plone api
    myproducts/                 # this is where you develop your Zope/Plone products
    build/                      # the directory where you built Zope and Python
    src/                        # the directory where all downloads get stored
    zeo_py2.3.5_zo2.8.1-final_pl2.1-rc3/
        Python-2.3.5             # a full python installation including ipython
        Zope-2.8.1-final         # a full zope installation
        thirdpartyproducts/     # all 3rd party products for both zeo clients are stored (symlinked) here,
                                # you shouldn't edit these when developing as you may break your upgrade path
        zeo_server/             # the zeo server that contains the shared ZODB
        zeo_client0/            # the zeo client that will act as our webserver on port 8080
        zeo_client1/            # the zeo client that will act as our interactive debugger (port 9080)

Now we need a filesystem product to debug, we will use the MySite product (mandatory reading for filesystem product development). Among other things the MySite Product subclasses Products.CMFPlone.MembershipTool and modifies the addMember function to send an email to newly registered users (whether they ask for it or not). This causes an exception on a fresh Plone site with no MailHost set up. This is what we will debug below. Install the MySite product by running the following commands.

cd ~/plonedev/src
wget -c http://www.neuroinf.de/LabTools/MySite-0_5.tgz
cd ~/plonedev/myproducts
tar -zxf ~/plonedev/src/MySite-0_5.tgz
cp -R MySite-0_5/* .
rm -rf MySite-0_5
cd ~/plonedev
./startPlone2.1-rc3.sh

then

Browse to http://localhost:8080/Acme
Log in as the site manager. username:manager, password:manager
Click on preferences - Add/Remove Products
Select MySite from the "Products available for install" and click the "install" button (you should also add the PloneErrorReporting tool while you're at it)
The MySite filesystem product is now installed
Logout
Click the join button
Fill in some registration details and click register
You should be presented with this screen

If you've got this far, you're all setup to start testing the various IDE's

Wing IDE Professional 2.0.3

Works really well, out of the box with Zope 2.8.1 and Plone 2.1-rc3 check out the documentation at http://wingware.com/doc/howtos/quickstart
http://wingware.com/doc/howtos/zope
http://wingware.com/doc/howtos/plone
The WingDBG product was installed for you by setupplonezeo.sh above, but you can get it from

wget -c http://wingware.com/pub/wingide/2.0.3/WingDBG-2.0.3-2.tar

Download and intall the evaluation version of wingide-2.0.3
start wingide
Close any existing projects
Open a new project
Project - add directory tree. Use your zope instance
```
~/plonedev/zeo_py2.3.5_zo2.8.1-final_pl2.1-rc3/zeo_client0
```
as the directory root (sit back and wait for a long time while wing analyzes all the source files for autocompletion purposes)
Project - Properties - Extensions - Enable Zope/Plone support - and add the INSTANCE_HOME of your zope instance (~/plonedev/zeo_py2.3.5_zo2.8.1-final_pl2.1-rc3/zeo_client0). This will cause your /etc/zope.conf file to be analyzed and will prompt you to add additional directories to your project (very cool).
Start Zope/Plone
```
~/plonedev/startPlone2.1-rc3.sh
```
open up the ZMI http://localhost:8080/manage
add the Wing Debugger product to your root folder
Give it a name/id e.g. WingDebugger
Click the change button
Click the ok button
Click start debugger
Open the file ~/plonedev/myproducts/MySite/MembershipTool.py in the wingide
set a breakpoint in the addMember function
Browse to http://localhost:50080/Acme
Click join, fill in some random details and then click register
The wing ide should pop up and there you go, you're off debugging!

BoaConstructor

Provides several ways to debug Zope/Plone as listed in it's documentation, click on help and search for zope Install Boa debugger extensions

Download and install Boa-Constructor 0.4.4

cd ~/plonedev/src
wget -c http://heanet.dl.sourceforge.net/sourceforge/boa-constructor/boa-constructor-0.4.4.zip
cd ~/plonedev
unzip ~/plonedev/src/boa-constructor-0.4.4.zip
cd boa-constructor-0.4.4
python Boa.py

Install the Boa Debug Server extension

cp boa-constructor-0.4.4/Debugger/BoaDebugServer.py zeo_py2.3.5_zo2.8.1-final_pl2.1-rc3/zeo_client0/Extensions/

Edit BoaDebugServer.py and replace c:/Path/To/Boa with the /home/YOURUSERNAME/plonedev/boa-constructor-0.4.4. You may also optionally replace the username, password passed to the start function for security. e.g.

   1:import sys
   2:
   3:# don't trace this module
   4:__traceable__ = 0
   5:
   6:def StartDebugServer():
   7:    sys.path.append('/home/YOURUSERNAME/plonedev/boa-constructor-0.4.4')
   8:
   9:    from Debugger.RemoteServer import start
  10:    start('', '') # username, password
  11:
  12:    return 'Debug Server started, attach to it from the IDE.'
  13:
  14:def HookDebugServer():
  15:    if hasattr(sys, 'debugger_control'):
  16:        sys.debugger_control.set_traceable()
  17:        sys.debugger_control.set_continue()
  18:    else:
  19:        raise Exception('Not running in the debugger.')
  20:
  21:    return 'Debug Server hooked, breakpoints now active.'

Start Zope/Plone
```
~/plonedev/startPlone2.1-rc3.sh
```
Open the ZMI and add the external methods
- Id=StartDebugServer, Title=StartDebugServer, Module Name=BoaDebugServer, Function Name=StartDebugServer
- Id=HookDebugServer, Title=HookDebugServer, Module Name=BoaDebugServer, Function Name=HookDebugServer
With a browser, execute the StartDebugServer method.
From the IDE open ~/plonedev/myproducts/MySkin/MembershipTool.py choose File->Attach to debugger.
Accept the defaults on the dialog box that pops up
Set a breakpoint in the addmember function on line 33 by double clicking in the gutter (you should see a red circle appear)
With a browser, execute the HookDebugServer method. The debugger is now hooked and ready and soft breakpoints should work.
Browse to http://localhost:8080/Acme/join_form and fill in any details, then click register
The browser should appear to hang, switch to Boa, you should be ready to step through the code from the breakpoint at line 33

There is also a BoaDebugger product which works well with Zope 2.7, however I could not get this product to work with Zope 2.8.

Emacs with pdb and IPython

This howto is directly lifted from http://docs.neuroinf.de/programming-plone/debug with a few small additions. First of all we need to setup emacs to syntax highlight and code complete python

If you haven't already got emacs install it e.g. on gentoo
```
emerge emacs
```

We want python syntax highlighting and code completion in emacs, so we need to download the appropriate emacs modes, run the following shell commands to get the modes

mkdir ~/elisp
cd ~/elisp
wget -c http://heanet.dl.sourceforge.net/sourceforge/python-mode/python-mode-1.0alpha.tar.gz
tar -zxvf python-mode-1.0alpha.tar.gz
wget -c http://scipy.net/cgi-bin/viewcvsx.cgi/*checkout*/ipython/doc/ipython.el?rev=HEAD&content-type=text/plain

edit the file ~/.emacs and add the following lines to it (if they aren't there already) to ensure your new python modes are installed

(global-font-lock-mode t)
(setq font-lock-maximum-decoration t)
(add-to-list 'load-path "~/elisp")
(load "~/elisp/python-mode-1.0alpha/python-mode.el")
(setq auto-mode-alist (cons '("\\.py$" . python-mode) auto-mode-alist))
(setq interpreter-mode-alist (cons '("python" . python-mode)  interpreter-mode-alist))
(autoload 'python-mode "python-mode" "Python editing mode." t)
(setq ipython-command "/usr/bin/ipython")
(require 'ipython)
(global-set-key [(f6)] 'ipython-complete)

The above commands are a once off emacs setup, you can test syntax highlighting and code completion in emacs as follows

open up a console and type
```
emacs -nw
```
type
```
<Alt> x py-shell
```

This will bring up the ipython shell, try this

import os
os. (press F6) you should see a list of completions (if you added the f6 keybinging command to your ~/.emacs file above)
os.system?? - should give you the doc string sfor os.system

Now on to debugging our Zope filesystem product

edit the file ~/plonedev/myproducts/MySite/MembershipTool.py and add the line import pdb; pdb.set_trace() to the addMember function (line 33) e.g.

20:class MembershipTool(BaseTool):
21:    
22:    meta_type = "Membership Tool"
23:    title="my custom membership tool"
24:
25:    security = ClassSecurityInfo()
26:
27:    security.declarePrivate('addMember')
28:    def addMember(self, id, password, roles, domains, properties=None):
29:        '''
30:        Adds a new member to the user folder.  Security checks will have
31:        already been performed.  Called by portal_registration.
32:        '''
33:        import pdb; pdb.set_trace()
34:        BaseTool.addMember(self, id, password, roles, domains, properties)
35:
36:        # plus my custom method
37:        self.notifyAdmin(id,properties)
38:
39:    security.declarePrivate('notifyAdmin')
40:    def notifyAdmin(self, member_id, properties):
41:        """
42:        Send email to site admin about the join
43:        This assumes that the mailhost and email_from_address

from a console start the zeo server

~/plonedev/zeo_py2.3.5_zo2.8.1-final_pl2.1-rc3/zeo_server/bin/zeoctl start

start emacs in a console
```
emacs -nw
```
open a shell in emacs
```
<Alt> x shell
```

run zope from the shell in emacs

cd ~/plonedev/zeo_py2.3.5_zo2.8.1-final_pl2.1-rc3/zeo_client0/bin
./runzope

Open your browser and browse to http://localhost:8080/Acme/join_form fill in some details, click register and ...
Switch to emacs, you should be looking at the python debugger in one buffer and some python source code in another buffer.
This pdb tutorial shows you what to do from here. In brief, at the pdb prompt:
type "u" to get to move up the stack to your source code.
typing "n" steps you through the code.
typing "l" lists the code around the current line.
typing "h" will show you the list of available commands.
typing "h some_command" will show you help for that command.
You can enter python variables and strings here to test their values.
emacs keeps the code synchronized with the stepping in pdb which is a very nice feature

PyDev 0.98 with eclipse 3.1

Download and install Eclipse 3.1, you will need to use Java 1.5 for PyDev
Install PyDev
Select Window - Preferences -PyDev - Python Interpreter - New and select ~/plonedev/zeo_py2.3.5_zo2.8.1-final_pl2.1-rc3/Python-2.3.5/bin/python
In eclipse, Add a simple project with ~/plonedev as the root directory, call the project plonedev, this will cause pydev to build your workspace for code completion purposes, this will take ages as there is alot of code (Python, Zope etc.)
edit the file ~/plonedev/myproducts/MySite/MembershipTool.py and add the line import pdb; pdb.set_trace() to the addMember function (line 33) as shown above
open the file debugPlone2.1-rc3.py in your editor, add a breakpoint to last line by double clicking on the margin
Right click on debugPlone2.1-rc3 in the Navigator view and select Debug As > Python Run
Eclipse will ask you do you want to switch to the Debug perspective (say yes), and you will be brought to a very well laid out debugging environment with the debugger stopped at the breakpoint you set
You can step in to your hearts content to see how Zope and Plone start up, but eventually, select Run - Resume to complete the server startup
Open your browser and browse to http://localhost:8080/Acme/join_form fill in some details, click register and ...
Switch to eclipse and you will see that the console has dropped in to the Python debugger (pdb)
Continue debugging as above with emacs, Unfortunately the source code in the editor doesn't follow pdb as emacs does, if anyone knows of a better way to do this, please let me know...

jEdit with JpyDbg

Download and install jEdit
Install Console Sidekick and JpyDbg, from jEdit, Plugins - Plugin Manager - Install (select all the plugins you need, very extensive list of excellent plugins (especially for xml)), note: it is important not to install the jython interpreter as this conflicts with JpyDbg in the sidekick structure browser.
Set up JpyDbg as described here
edit the file ~/plonedev/myproducts/MySite/MembershipTool.py (as shown above) and add the line import pdb; pdb.set_trace() to the addMember function (line 33)
Open the file ~/plonedev/debugPlone2.1-rc3.py in your editor
Utilities - Global Options - Docking, dock the Python Environment and console on the bottom, and dock the structure browser on the left, you should see the structure of your python file in the structure browser now
Click the Python Environment button at the bottom of the screen to view the python environment
Set a breakpoint on the last line of ~/plonedev/debugPlone2.1-rc3.py by clicking in the left margin
Click the green triangle button on the python environment to start debugging
Open your browser and browse to http://localhost:8080/Acme/join_form fill in some details, click register and ...
It is now possible to use the python debugger (pdb) in the python environment console, however newlines seem to be ignored and the source code does not follow the pdb as emacs does. You should however still be able to step through your code as described in this pdb tutorial

Summary

IDE	Free	Zope Integration
Wing IDE	No	Yes
Boa Constructor	Yes	Yes
Emacs	Yes	No
Eclipse with PyDev	Yes	No
jEdit	Yes	No

Michael Thornhill

Friday, September 02, 2005

How to debug Zope/Plone with an IDE