Tuesday, November 15, 2005

RecentBufferSwitcher for jEdit

I've been using jEdit for a long time now at work, it's a great cross platform easily extensible text editor. My only problem with it was that I couldn't switch to recently used buffers easily so I had a go at writing a macro (in jython) to switch buffers (this was hacked from code copied from Ollie Rutherfurds examples). This worked well but wasn't the speediest so I had a go at writing a plugin to do this.

This turned out to be easy enough as there is good documentation provided on writing plugins. So I wrote the RecentBufferSwitcher plugin which was released on jEdit plugin manager today... yeaaaah!

Friday, September 02, 2005

How to debug Zope/Plone with an IDE

This post gives a quick introduction on how to debug Filesystem Products in Zope/Plone with the following IDEs
  1. Wing IDE Professional 2.0.3
  2. Boa Constructor 0.4.4
  3. Eclipse 3.1 with PyDev 0.98
  4. Emacs 21.4.1 with pdb and ipython 0.6.15
  5. jEdit 4.2final with JpyDbg 0.9.1
If anyone knows of any other suitable IDEs (or if I've left out features in the ones above), please feel free to let me know in the comment section at the bottom of this post. This post doesn't aim to say which IDE is better than another, just to show how to use them for debugging.
This post is a follow on from Team Development with Plone/ Zope/ ZEO/ Subversion/ ipython. You will need to run these commands to try out the IDE examples in this post.
mkdir ~/plonedev
cd ~/plonedev
wget -c http://homepage.eircom.net/~rachra/setupplonezeo.sh
wget -c http://homepage.eircom.net/~rachra/plonehelper.py
chmod +x setupplonezeo.sh
./setupplonezeo.sh 1 Acme
Running the setupplonezeo.sh script will leave you with the directory structure shown below
~/plonedev/    
    startPlone2.1-rc3.sh        # this script starts the zeo server on port 8100 and plone listening on port 8080
    startIPythonPlone2.1-rc3.sh # this script starts an IPython interpreter on your zeo_client1 (port 9080)
    debugPlone2.1-rc3.py        # a python script to start zeo_client0 (port 8080)
    plonehelper.py              # python helper script for creating a plone instance
                                # installing products and selecting a skin using the plone api
    myproducts/                 # this is where you develop your Zope/Plone products
    build/                      # the directory where you built Zope and Python
    src/                        # the directory where all downloads get stored
    zeo_py2.3.5_zo2.8.1-final_pl2.1-rc3/
        Python-2.3.5             # a full python installation including ipython
        Zope-2.8.1-final         # a full zope installation
        thirdpartyproducts/     # all 3rd party products for both zeo clients are stored (symlinked) here,
                                # you shouldn't edit these when developing as you may break your upgrade path
        zeo_server/             # the zeo server that contains the shared ZODB
        zeo_client0/            # the zeo client that will act as our webserver on port 8080
        zeo_client1/            # the zeo client that will act as our interactive debugger (port 9080)
Now we need a filesystem product to debug, we will use the MySite product (mandatory reading for filesystem product development). Among other things the MySite Product subclasses Products.CMFPlone.MembershipTool and modifies the addMember function to send an email to newly registered users (whether they ask for it or not). This causes an exception on a fresh Plone site with no MailHost set up. This is what we will debug below. Install the MySite product by running the following commands.
cd ~/plonedev/src
wget -c http://www.neuroinf.de/LabTools/MySite-0_5.tgz
cd ~/plonedev/myproducts
tar -zxf ~/plonedev/src/MySite-0_5.tgz
cp -R MySite-0_5/* .
rm -rf MySite-0_5
cd ~/plonedev
./startPlone2.1-rc3.sh
then
  1. Browse to http://localhost:8080/Acme
  2. Log in as the site manager. username:manager, password:manager
  3. Click on preferences - Add/Remove Products
  4. Select MySite from the "Products available for install" and click the "install" button (you should also add the PloneErrorReporting tool while you're at it)
  5. The MySite filesystem product is now installed
  6. Logout
  7. Click the join button
  8. Fill in some registration details and click register
  9. You should be presented with this screen
If you've got this far, you're all setup to start testing the various IDE's

Wing IDE Professional 2.0.3

Works really well, out of the box with Zope 2.8.1 and Plone 2.1-rc3 check out the documentation at http://wingware.com/doc/howtos/quickstart
http://wingware.com/doc/howtos/zope
http://wingware.com/doc/howtos/plone
The WingDBG product was installed for you by setupplonezeo.sh above, but you can get it from
wget -c http://wingware.com/pub/wingide/2.0.3/WingDBG-2.0.3-2.tar
  1. Download and intall the evaluation version of wingide-2.0.3
  2. start wingide
  3. Close any existing projects
  4. Open a new project
  5. Project - add directory tree. Use your zope instance
    ~/plonedev/zeo_py2.3.5_zo2.8.1-final_pl2.1-rc3/zeo_client0
    
    as the directory root (sit back and wait for a long time while wing analyzes all the source files for autocompletion purposes)
  6. Project - Properties - Extensions - Enable Zope/Plone support - and add the INSTANCE_HOME of your zope instance (~/plonedev/zeo_py2.3.5_zo2.8.1-final_pl2.1-rc3/zeo_client0). This will cause your /etc/zope.conf file to be analyzed and will prompt you to add additional directories to your project (very cool).
  7. Start Zope/Plone
    ~/plonedev/startPlone2.1-rc3.sh
    
  8. open up the ZMI http://localhost:8080/manage
  9. add the Wing Debugger product to your root folder
  10. Give it a name/id e.g. WingDebugger
  11. Click the change button
  12. Click the ok button
  13. Click start debugger
  14. Open the file ~/plonedev/myproducts/MySite/MembershipTool.py in the wingide
  15. set a breakpoint in the addMember function
  16. Browse to http://localhost:50080/Acme
  17. Click join, fill in some random details and then click register
  18. The wing ide should pop up and there you go, you're off debugging!
  19. Try click on the stack trace and browse to (~/plonedev//lib/python/ZPublisher/)Publish.py, line 386, put a breakpoint here, you should get into the debugger on every plone request now

BoaConstructor

Provides several ways to debug Zope/Plone as listed in it's documentation, click on help and search for zope Install Boa debugger extensions
  1. Download and install Boa-Constructor 0.4.4
    cd ~/plonedev/src
    wget -c http://heanet.dl.sourceforge.net/sourceforge/boa-constructor/boa-constructor-0.4.4.zip
    cd ~/plonedev
    unzip ~/plonedev/src/boa-constructor-0.4.4.zip
    cd boa-constructor-0.4.4
    python Boa.py
    
  2. Install the Boa Debug Server extension
    cp boa-constructor-0.4.4/Debugger/BoaDebugServer.py zeo_py2.3.5_zo2.8.1-final_pl2.1-rc3/zeo_client0/Extensions/
    
  3. Edit BoaDebugServer.py and replace c:/Path/To/Boa with the /home/YOURUSERNAME/plonedev/boa-constructor-0.4.4. You may also optionally replace the username, password passed to the start function for security. e.g.
       1:import sys
       2:
       3:# don't trace this module
       4:__traceable__ = 0
       5:
       6:def StartDebugServer():
       7:    sys.path.append('/home/YOURUSERNAME/plonedev/boa-constructor-0.4.4')
       8:
       9:    from Debugger.RemoteServer import start
      10:    start('', '') # username, password
      11:
      12:    return 'Debug Server started, attach to it from the IDE.'
      13:
      14:def HookDebugServer():
      15:    if hasattr(sys, 'debugger_control'):
      16:        sys.debugger_control.set_traceable()
      17:        sys.debugger_control.set_continue()
      18:    else:
      19:        raise Exception('Not running in the debugger.')
      20:
      21:    return 'Debug Server hooked, breakpoints now active.'
    
  4. Start Zope/Plone
    ~/plonedev/startPlone2.1-rc3.sh
    
  5. Open the ZMI and add the external methods
    • Id=StartDebugServer, Title=StartDebugServer, Module Name=BoaDebugServer, Function Name=StartDebugServer
    • Id=HookDebugServer, Title=HookDebugServer, Module Name=BoaDebugServer, Function Name=HookDebugServer
  6. With a browser, execute the StartDebugServer method.
  7. From the IDE open ~/plonedev/myproducts/MySkin/MembershipTool.py choose File->Attach to debugger.
  8. Accept the defaults on the dialog box that pops up
  9. Set a breakpoint in the addmember function on line 33 by double clicking in the gutter (you should see a red circle appear)
  10. With a browser, execute the HookDebugServer method. The debugger is now hooked and ready and soft breakpoints should work.
  11. Browse to http://localhost:8080/Acme/join_form and fill in any details, then click register
  12. The browser should appear to hang, switch to Boa, you should be ready to step through the code from the breakpoint at line 33
There is also a BoaDebugger product which works well with Zope 2.7, however I could not get this product to work with Zope 2.8.

Emacs with pdb and IPython

This howto is directly lifted from http://docs.neuroinf.de/programming-plone/debug with a few small additions. First of all we need to setup emacs to syntax highlight and code complete python
  1. If you haven't already got emacs install it e.g. on gentoo
    emerge emacs
    
  2. We want python syntax highlighting and code completion in emacs, so we need to download the appropriate emacs modes, run the following shell commands to get the modes
    mkdir ~/elisp
    cd ~/elisp
    wget -c http://heanet.dl.sourceforge.net/sourceforge/python-mode/python-mode-1.0alpha.tar.gz
    tar -zxvf python-mode-1.0alpha.tar.gz
    wget -c http://scipy.net/cgi-bin/viewcvsx.cgi/*checkout*/ipython/doc/ipython.el?rev=HEAD&content-type=text/plain
    
  3. edit the file ~/.emacs and add the following lines to it (if they aren't there already) to ensure your new python modes are installed
    (global-font-lock-mode t)
    (setq font-lock-maximum-decoration t)
    (add-to-list 'load-path "~/elisp")
    (load "~/elisp/python-mode-1.0alpha/python-mode.el")
    (setq auto-mode-alist (cons '("\\.py$" . python-mode) auto-mode-alist))
    (setq interpreter-mode-alist (cons '("python" . python-mode)  interpreter-mode-alist))
    (autoload 'python-mode "python-mode" "Python editing mode." t)
    (setq ipython-command "/usr/bin/ipython")
    (require 'ipython)
    (global-set-key [(f6)] 'ipython-complete)
    
The above commands are a once off emacs setup, you can test syntax highlighting and code completion in emacs as follows
  1. open up a console and type
    emacs -nw
    
  2. type
    <Alt> x py-shell
    
  3. This will bring up the ipython shell, try this
    import os
    os. (press F6) you should see a list of completions (if you added the f6 keybinging command to your ~/.emacs file above)
    os.system?? - should give you the doc string sfor os.system
    
Now on to debugging our Zope filesystem product
  1. edit the file ~/plonedev/myproducts/MySite/MembershipTool.py and add the line import pdb; pdb.set_trace() to the addMember function (line 33) e.g.
    20:class MembershipTool(BaseTool):
    21:    
    22:    meta_type = "Membership Tool"
    23:    title="my custom membership tool"
    24:
    25:    security = ClassSecurityInfo()
    26:
    27:    security.declarePrivate('addMember')
    28:    def addMember(self, id, password, roles, domains, properties=None):
    29:        '''
    30:        Adds a new member to the user folder.  Security checks will have
    31:        already been performed.  Called by portal_registration.
    32:        '''
    33:        import pdb; pdb.set_trace()
    34:        BaseTool.addMember(self, id, password, roles, domains, properties)
    35:
    36:        # plus my custom method
    37:        self.notifyAdmin(id,properties)
    38:
    39:    security.declarePrivate('notifyAdmin')
    40:    def notifyAdmin(self, member_id, properties):
    41:        """
    42:        Send email to site admin about the join
    43:        This assumes that the mailhost and email_from_address
    
  2. from a console start the zeo server
    ~/plonedev/zeo_py2.3.5_zo2.8.1-final_pl2.1-rc3/zeo_server/bin/zeoctl start
    
  3. start emacs in a console
    emacs -nw
    
  4. open a shell in emacs
    <Alt> x shell
    
  5. run zope from the shell in emacs
    cd ~/plonedev/zeo_py2.3.5_zo2.8.1-final_pl2.1-rc3/zeo_client0/bin
    ./runzope
    
  6. Open your browser and browse to http://localhost:8080/Acme/join_form fill in some details, click register and ...
  7. Switch to emacs, you should be looking at the python debugger in one buffer and some python source code in another buffer.
  8. This pdb tutorial shows you what to do from here. In brief, at the pdb prompt:
    type "u" to get to move up the stack to your source code.
    typing "n" steps you through the code.
    typing "l" lists the code around the current line.
    typing "h" will show you the list of available commands.
    typing "h some_command" will show you help for that command.
    You can enter python variables and strings here to test their values.
  9. emacs keeps the code synchronized with the stepping in pdb which is a very nice feature

PyDev 0.98 with eclipse 3.1

  1. Download and install Eclipse 3.1, you will need to use Java 1.5 for PyDev
  2. Install PyDev
  3. Select Window - Preferences -PyDev - Python Interpreter - New and select ~/plonedev/zeo_py2.3.5_zo2.8.1-final_pl2.1-rc3/Python-2.3.5/bin/python
  4. In eclipse, Add a simple project with ~/plonedev as the root directory, call the project plonedev, this will cause pydev to build your workspace for code completion purposes, this will take ages as there is alot of code (Python, Zope etc.)
  5. edit the file ~/plonedev/myproducts/MySite/MembershipTool.py and add the line import pdb; pdb.set_trace() to the addMember function (line 33) as shown above
  6. open the file debugPlone2.1-rc3.py in your editor, add a breakpoint to last line by double clicking on the margin
  7. Right click on debugPlone2.1-rc3 in the Navigator view and select Debug As > Python Run
  8. Eclipse will ask you do you want to switch to the Debug perspective (say yes), and you will be brought to a very well laid out debugging environment with the debugger stopped at the breakpoint you set
  9. You can step in to your hearts content to see how Zope and Plone start up, but eventually, select Run - Resume to complete the server startup
  10. Open your browser and browse to http://localhost:8080/Acme/join_form fill in some details, click register and ...
  11. Switch to eclipse and you will see that the console has dropped in to the Python debugger (pdb)
  12. Continue debugging as above with emacs, Unfortunately the source code in the editor doesn't follow pdb as emacs does, if anyone knows of a better way to do this, please let me know...

jEdit with JpyDbg

  1. Download and install jEdit
  2. Install Console Sidekick and JpyDbg, from jEdit, Plugins - Plugin Manager - Install (select all the plugins you need, very extensive list of excellent plugins (especially for xml)), note: it is important not to install the jython interpreter as this conflicts with JpyDbg in the sidekick structure browser.
  3. Set up JpyDbg as described here
  4. edit the file ~/plonedev/myproducts/MySite/MembershipTool.py (as shown above) and add the line import pdb; pdb.set_trace() to the addMember function (line 33)
  5. Open the file ~/plonedev/debugPlone2.1-rc3.py in your editor
  6. Utilities - Global Options - Docking, dock the Python Environment and console on the bottom, and dock the structure browser on the left, you should see the structure of your python file in the structure browser now
  7. Click the Python Environment button at the bottom of the screen to view the python environment
  8. Set a breakpoint on the last line of ~/plonedev/debugPlone2.1-rc3.py by clicking in the left margin
  9. Click the green triangle button on the python environment to start debugging
  10. Open your browser and browse to http://localhost:8080/Acme/join_form fill in some details, click register and ...
  11. It is now possible to use the python debugger (pdb) in the python environment console, however newlines seem to be ignored and the source code does not follow the pdb as emacs does. You should however still be able to step through your code as described in this pdb tutorial

Summary

IDEFreeZope Integration
Wing IDENoYes
Boa ConstructorYesYes
EmacsYesNo
Eclipse with PyDevYesNo
jEditYesNo

Thursday, August 18, 2005

Team Development with Plone/ Zope/ ZEO/ Subversion/ ipython

Introduction
Team Development with Zope (7) can be problematic (1). The ZODB is a large binary file (var/Data.fs) that stores all information about a Zope instance, the ZODB is opaque to all but python. The ZODB is typically accessed Through The Web (TTW) using the Zope Management Interface (ZMI) typically http://localhost:8080/manage. All data controlled and published by Zope are held in this ZODB, while almost all available Source Control Management (SCM) and replication tools needed for version management and staging require a traditional file system representation of data. Though there is a simple version control mechanism built-in to the ZODB (one development line for single objects), this soon proves insufficient even for small applications (4). This post shows how to set up a zeo server with 2 clients, one acting as a web server, the other as an ipython interpreter (this gives us access to the ZODB object for debugging purposes) The following are typical requirements for Team Development in Plone:
  1. Use Source Control (CVS, Subversion, etc.).
  2. Setup environment so debugging is possible (11)
  3. File system based.
  4. Don't modify Zope or Plone through the ZMI so that different versions of Zope/Plone can be setup and tested without requiring TTW intervention.
  5. Allow off line development (Don't use a central development server)
Problems with Team Development in Zope
  1. Objects (Python Scripts, Images ZSQL Methods, etc.) are stored as binary objects in the Zope Object Database (ZODB).
  2. Can't use file system utilities such as grep, find for development.
  3. Editing documents is painful if your "favorite editor" doesn't work with the external editor tool (9).
  4. Can't use source control management, e.g. CVS, Subversion, on objects in the ZODB (exception: (CVSFile) and (ExternalFile) provide CVS through the ZMI).
  5. Can't write test suites for binary objects in database.
Possible solution
  1. Do not use ZMI at all ever!, use Filesystem Products (2) (6) (16).
    • Build products for each major feature (19) (20)
    • Build a site product for site customization (16)
    • Avoid disconnected External methods, scripts etc.
  2. Write scripts for creating your Plone instance and configuration (involves learning Zope/Plone api), commit these scripts to SCM.
While point 1 is covered well in the literature, it is not so easy to find scripts for setting up a Plone instance. This hack shows a bash script that sets up a Plone instance from a script and which installs the MySkin product into this Plone instance. This gives us the starting point for File system based Plone development. To execute this script and build plone 2.1 do the following (on linux, I use Gentoo and gcc 3.4.4).
mkdir ~/plonedev
cd ~/plonedev
wget -c http://homepage.eircom.net/~rachra/setupplonezeo.sh
wget -c http://homepage.eircom.net/~rachra/plonehelper.py
chmod +x setupplonezeo.sh
./setupplonezeo.sh 1 Acme
This should (all going well) leave you with the following directory structure
~/plonedev/    
    startPlone2.1-rc3.sh        # this script starts the zeo server on port 8100 and plone listening on port 8080
    startIPythonPlone2.1-rc3.sh # this script starts an IPython interpreter on your zeo_client1 (port 9080)
    debugPlone2.1-rc3.py        # a python script to start zeo_client0 (port 8080)
    plonehelper.py              # python helper script for creating a plone instance
                                # installing products and selecting a skin using the plone api
    myproducts/                 # this is where you develop your Zope/Plone products
    build/                      # the directory where you built Zope and Python
    src/                        # the directory where all downloads get stored
    zeo_py2.3.5_zo2.8.1-final_pl2.1-rc3/
        Python-2.3.5/            # a full python installation including ipython
        Zope-2.8.1-final/        # a full zope installation
        thirdpartyproducts/     # all 3rd party products for both zeo clients are stored (symlinked) here,
                                # you shouldn't edit these when developing as you may break your upgrade path
        zeo_server/             # the zeo server that contains the shared ZODB
        zeo_client0/            # the zeo client that will act as our webserver on port 8080
        zeo_client1/            # the zeo client that will act as our interactive debugger (port 9080)
To start the zeo server running on port 8100 and plone listening on port 8080 run the following
./startPlone2.1-rc3.sh
To start an ipython prompt with the ZODB app object in your namespace, run the following:
./startIPythonPlone2.1-rc3.sh
The script also builds a demo Plone instance called Acme with a custom filesystemskin located in the products directory at AcmeSkin (you should see that the plone logo has been replaced when you browse to http://localhost:8080/Acme. IPython can be set up for different colour console types. I had to edit ~/.ipython/ipythonrc as follows for my white background gnome-terminal,
115:# colors - Coloring option for prompts and traceback printouts.
116:
117:# Currently available schemes: NoColor, Linux, LightBG.
118:
119:# This option allows coloring the prompts and traceback printouts. This
120:# requires a terminal which can properly handle color escape sequences. If you
121:# are having problems with this, use the NoColor scheme (uses no color escapes
122:# at all).
123:
124:# The Linux option works well in linux console type environments: dark
125:# background with light fonts.
126:
127:# LightBG is similar to Linux but swaps dark/light colors to be more readable
128:# in light background terminals.
129:
130:# keep uncommented only the one you want:
131:#colors Linux
132:colors LightBG
133:#colors NoColor
134:
Try the following commands in ipython to get a feel for the plone api (make sure to test the TAB button for auto-completion.)
from Products.CMFPlone import transaction
for id, ref in app.objectItems(): print "%-25s: %s"% (id, repr(ref))
app.Acme.objectIds??
app.Acme.objectIds()
for k,v in app.Acme.contentItems(): print "%-11s: %s"%(k,repr(v))

# try out any of your own commands here, 
# DocFinderTab is installed and viewable through the ZMI for help with the Zope api

# to synchronize with the ZODB
app._p_jar.sync()
# to commit your changes to the database
transaction.commit()
Ctrl-d gets you out of the IPython prompt. Any changes you make to filesystem products are picked up automatically (if there is a refresh.txt file in the Products root directory) as Zope is setup here to run with debug-mode on. Don't forget to run ./stopPlone2.1-rc3.sh when you are finished to close the ZEO server and the Zope server. You can also use this script to set up a Plone 2.0.5 development environment e.g. to setup Plone 2.0.5 with a PloneId of TestSite on port 8888
./setupplonezeo.sh 0 TestSite 8888
This would leave you with the following directory structure (assuming you ran the command "./setupplonezeo.sh 1" previously)
~/plonedev/
    startPlone2.1-rc3.sh        # this script starts the zeo server on port 8100 and plone listening on port 8080
    startPlone2.0.5.sh          # this script starts the zeo server on port 9999 abd plone listening on port 8888
    startIPythonPlone2.1-rc3.sh # this script starts an IPython interpreter on your zeo_client1 (port 9080)    
    startIPythonPlone2.0.5.sh   # this script starts an IPython interpreter on your zeo_client1 (port 9888)
    debugPlone2.1-rc3.py        # a python script to start zeo_client0 (port 8080)
    debugPlone2.0.5.py          # a python script to start zeo_client0 (port 9080)
    plonehelper.py              # python helper script for creating a plone instance
                                # installing products and selecting a skin using the plone api
    myproducts/                 # this is where you develop your Zope/Plone products
    build/                      # the directory where you built Zope and Python
    src/                        # the directory where all downloads get stored
    zeo_py2.3.5_zo2.8.1-final_pl2.1-rc3/
        Python-2.3.5/            # a full python installation including ipython
        Zope-2.8.1-final/        # a full zope installation
        thirdpartyproducts/     # all 3rd party products for both zeo clients are stored (symlinked) here,
                                # you shouldn't edit these when developing as you may break your upgrade path
        zeo_server/             # the zeo server that contains the shared ZODB
        zeo_client0/            # the zeo client that will act as our webserver on port 8080
        zeo_client1/            # the zeo client that will act as our interactive debugger (port 9080)
    zeo_py2.3.4_zo2.7.4_pl2.0.5/
        Python2.3.4/            # a full python installation including ipython
        Zope-2.7.4-0/           # a full zope installation
        thirdpartyproducts/     # all 3rd party products for both zeo clients are stored (symlinked) here,
                                # you shouldn't edit these when developing as you may break your upgrade path
        zeo_server/             # the zeo server that contains the shared ZODB
        zeo_client0/            # the zeo client that will act as our webserver on port 8888
        zeo_client1/            # the zeo client that will act as our interactive debugger (port 9888)
This setup has the advantage that you can easily and quickly test your Zope products across two versions of Plone. If you get locking errors and ipython doesn't start you need to increase the sleep time to allow servers to start, e.g. to increase the sleep time to 63 seconds
./setupplonezeo.sh 0 TestSite 8888 63
References
  1. Why not to use Zope: http://www.amk.ca/python/writing/why-not-zope.html
  2. Plone best practices: https://plone.org/documentation/tutorial/best-practices/
  3. Team development with Plone/Zope: http://www.zope.org/Members/k_vertigo/Stories/TeamZope
  4. zsync: http://www.elegosoft.com/index_zync.html
  5. Unit testing plone: http://www.zope.org/Members/shh/Tutorial/PloneTestCase.pdf
  6. Scripting plone: http://docs.neuroinf.de/programming-plone
  7. Beginners guide to Zope/Plone: http://www.neuroinf.de/Miscellaneous/BeginnersGuide
  8. Introduction to Plone: http://docs.neuroinf.de/PloneBook
  9. Zope External Editor: http://www.zope.org/Members/Caseman/ExternalEditor
  10. Zope Test Case: http://www.zope.org/Members/shh/ZopeTestCase
  11. Debugging Zope: http://plone.org/Members/pupq/debug
  12. Debugging Zope: http://www.zope.org/Members/klm/ZopeDebugging/ConversingWithZope
  13. ZEO and debugging: http://www.zope.org/Members/dshaw/AdvancedSiteSetup
  14. Installing ZEO: http://www.plope.com/Books/2_7Edition/ZEO.stx
  15. Debugging Zope: http://zopewiki.org/DebuggingZopeWithPythonDebugger2
  16. MySkin Plone customization: http://www.ucl.ac.uk/is/fiso/engsciences/tutorials/plone/plone_pages/customise_plone2.htm
  17. Archetypes Development environment setup: http://plone.org/documentation/archetypes/wiki/SettingUpArchetypesProductDevelopmentEnvironment
  18. Archetypes home page: http://plone.org/documentation/archetypes
  19. Archetypes developers guide: http://plone.org/documentation/archetypes/ArchetypesDeveloperGuide
  20. ArchGenXML: http://plone.org/documentation/tutorial/archgenxml-getting-started
  21. Robust Plone installation with ZEO: http://plone.org/documentation/tutorial/robust-installation
  22. ipython: http://ipython.scipy.org/
  23. ipython tutorial: http://www.onlamp.com/pub/a/python/2005/01/27/ipython.html
  24. BoaDebugger: http://plone.org/documentation/how-to/how-to-debug-products-with-boa-constructor
  25. Installing Zope and Pone http://docs.neuroinf.de/programming-plone/appendixC#1-5
Scripts
setupplonezeo.sh
   1:#!/bin/bash
   2:E_NOARGS=65
   3:PloneId="Acme"
   4:ZOPE_PORT=8080
   5:SLEEPTIME=20 
   6:if [ -z "$1" ]
   7:then
   8:  echo "Usage: `basename $0` [plone-version, 0 or 1] [ploneid, default $PloneId] [zope-port, default $ZOPE_PORT] [sleep-time, default $SLEEPTIME]"
   9:  echo "e.g. to build Plone 2.1 with a PloneId of TestSite allowing 15 seconds for server to start on port 8888"
  10:  echo "`basename $0` 1 TestSite 8888 15"
  11:  exit $E_NOARGS
  12:fi
  13:#set -x
  14:
  15:base=`pwd`
  16:mkdir $base/{src,build}
  17:cd $base/src
  18:if [ "$1" -ne "0" ]
  19:then
  20:    PythonVersion="2.3.5"
  21:    ZopeVersion="2.8.1-final"
  22:    PloneVersion="2.1-rc3"
  23:    ZEO_PORT=8100    
  24:    wget -c http://www.zope.org/Products/Zope/2.8.1/Zope-$ZopeVersion.tgz    
  25:else    
  26:    PythonVersion="2.3.4"
  27:    ZopeVersion="2.7.4-0"
  28:    PloneVersion="2.0.5"
  29:    ZEO_PORT=9999    
  30:    wget -c http://www.zope.org/Products/Zope/2.7.4/Zope-$ZopeVersion.tgz    
  31:fi
  32:StartPloneScript=startPlone$PloneVersion.sh
  33:StopPloneScript=stopPlone$PloneVersion.sh
  34:StartIPythonScript=startIPython$PloneVersion.sh
  35:PythonDebugScript=debugPlone$PloneVersion.py
  36:if [ -n "$2" ]
  37:then 
  38:    PloneId=$2
  39:fi
  40:if [ -n "$3" ]
  41:then 
  42:    ZOPE_PORT=$3
  43:fi
  44:if [ -n "$4" ]
  45:then 
  46:    SLEEPTIME=$4
  47:fi
  48:PloneTitle=$PloneId
  49:PloneDescription="CMS Prototype"
  50:SkinName="${PloneId}Skin"
  51:INSTALL_HOME=$base/zeo_py${PythonVersion}_zo${ZopeVersion}_pl$PloneVersion
  52:# remove existing install
  53:rm -rf $INSTALL_HOME
  54:PYTHON_HOME=$INSTALL_HOME/Python-$PythonVersion
  55:ZOPE_HOME=$INSTALL_HOME/Zope-$ZopeVersion
  56:PYTHON=$PYTHON_HOME/bin/python
  57:PRODUCTS_DIR=$INSTALL_HOME/thirdpartyproducts
  58:MYPRODUCTS_DIR=$base/myproducts
  59:ZEOSERVER_DIR=$INSTALL_HOME/zeo_server
  60:ZEOCLIENT0_DIR=$INSTALL_HOME/zeo_client0
  61:ZEOCLIENT1_DIR=$INSTALL_HOME/zeo_client1
  62:INSTANCE_USER=manager
  63:INSTANCE_PASSWORD=manager
  64:
  65:# Get Python, Zope, Plone, MySkin, ipython and a sample logo
  66:wget -c http://www.python.org/ftp/python/$PythonVersion/Python-$PythonVersion.tgz
  67:wget -c http://heanet.dl.sourceforge.net/sourceforge/plone/Plone-$PloneVersion.tar.gz
  68:wget -c http://plone.org/products/myskin/releases/0.1/MySkin-0.1.tar.gz
  69:wget -c http://www.zope.org/Members/shh/DocFinderTab/0.5.2/DocFinderTab-0.5.2.tar.gz
  70:wget -c  http://homepage.eircom.net/~rachra/acme.jpg
  71:wget -c http://ipython.scipy.org/dist/ipython-0.6.15.tar.gz
  72:wget -c http://wingware.com/pub/wingide/2.0.3/WingDBG-2.0.3-2.tar
  73:
  74:# build Python
  75:cd $base/build
  76:tar -zxf $base/src/Python-$PythonVersion.tgz
  77:cd Python-$PythonVersion
  78:./configure --prefix=$PYTHON_HOME --enable-unicode=ucs4
  79:make 
  80:make install
  81:
  82:# install ipython
  83:cd $base/build
  84:tar -zxvf $base/src/ipython-0.6.15.tar.gz
  85:cd ipython-0.6.15
  86:$PYTHON setup.py install
  87:
  88:# build Zope
  89:cd $base/build
  90:tar -zxf $base/src/Zope-$ZopeVersion.tgz
  91:cd Zope-$ZopeVersion
  92:./configure --prefix=$ZOPE_HOME --with-python=$PYTHON 
  93:make 
  94:make install
  95:
  96:# make a zeo server instance
  97:$PYTHON $ZOPE_HOME/bin/mkzeoinstance.py $ZEOSERVER_DIR
  98:
  99:# make 2 zeo client instances
 100:$PYTHON $ZOPE_HOME/bin/mkzopeinstance.py -d $ZEOCLIENT0_DIR -u $INSTANCE_USER:$INSTANCE_PASSWORD
 101:$PYTHON $ZOPE_HOME/bin/mkzopeinstance.py -d $ZEOCLIENT1_DIR -u $INSTANCE_USER:$INSTANCE_PASSWORD
 102:
 103:# unzip Plone and copy into Products directory
 104:mkdir $PRODUCTS_DIR
 105:cd $base/build
 106:tar -zxf $base/src/Plone-$PloneVersion.tar.gz
 107:cd Plone-$PloneVersion
 108:cp -R * $PRODUCTS_DIR
 109:
 110:# install docfinder
 111:cd $PRODUCTS_DIR
 112:tar -zxf $base/src/DocFinderTab-0.5.2.tar.gz
 113:
 114:# install WingDebugger
 115:tar -xf $base/src/WingDBG-2.0.3-2.tar
 116:
 117:# install our own filesystem skin, this would come from source control normally
 118:# install MySkin Product
 119:mkdir $MYPRODUCTS_DIR
 120:cd $MYPRODUCTS_DIR
 121:tar -zxf $base/src/MySkin-0.1.tar.gz
 122:
 123:# move MySkin directories to $SkinName
 124:mv MySkin $SkinName
 125:mv $SkinName/skins/MySkin $SkinName/skins/$SkinName
 126:
 127:# Convert all instances of MySkin to $SkinName in all files
 128:cd $SkinName
 129:find . -exec  sed  -i -e "s/MySkin/$SkinName/g" {} \;
 130:
 131:# copy a logo in here to test if filesystem skin is working
 132:cp $base/src/acme.jpg $MYPRODUCTS_DIR/$SkinName/skins/$SkinName/logo.jpg
 133:
 134:# symlink Product directories between clients
 135:rm -rf $ZEOCLIENT0_DIR/Products
 136:ln -s $PRODUCTS_DIR $ZEOCLIENT0_DIR/Products
 137:rm -rf $ZEOCLIENT1_DIR/Products
 138:ln -s $PRODUCTS_DIR $ZEOCLIENT1_DIR/Products
 139:
 140:# write out the zope.conf files for each client
 141:# client0 listens on port $ZOPE_PORT
 142:cat << EOF > $ZEOCLIENT0_DIR/etc/zope.conf
 143:%define INSTANCE $ZEOCLIENT0_DIR
 144:%define ZOPE $ZOPE_HOME
 145:instancehome \$INSTANCE
 146:debug-mode on
 147:products $MYPRODUCTS_DIR
 148:<eventlog>
 149:  level info
 150:  <logfile>
 151:    path \$INSTANCE/log/event.log
 152:    level info
 153:  </logfile>
 154:</eventlog>
 155:<logger access>
 156:  level WARN
 157:  <logfile>
 158:    path \$INSTANCE/log/Z2.log
 159:    format %(message)s
 160:  </logfile>
 161:</logger>
 162:<http-server>
 163:  # valid keys are "address" and "force-connection-close"
 164:  address $ZOPE_PORT
 165:  # force-connection-close on
 166:</http-server>
 167:<zodb_db temporary>
 168:    # Temporary storage database (for sessions)
 169:    <temporarystorage>
 170:      name temporary storage for sessioning
 171:    </temporarystorage>
 172:    mount-point /temp_folder
 173:    container-class Products.TemporaryFolder.TemporaryContainer
 174:</zodb_db>
 175:<zodb_db main>
 176:   mount-point /
 177:   # ZODB cache, in number of objects
 178:   cache-size 5000
 179:   <zeoclient>
 180:     server localhost:$ZEO_PORT
 181:     storage 1
 182:     name zeostorage
 183:     var \$INSTANCE/var
 184:     # ZEO client cache, in bytes
 185:     cache-size 20MB
 186:     # Uncomment to have a persistent disk cache
 187:     #client zeo1
 188:   </zeoclient>
 189:</zodb_db>
 190:EOF
 191:
 192:# client1 listens on port $ZOPE_PORT+1000
 193:cat << EOF > $ZEOCLIENT1_DIR/etc/zope.conf
 194:%define INSTANCE $ZEOCLIENT1_DIR
 195:%define ZOPE $ZOPE_HOME
 196:instancehome \$INSTANCE
 197:debug-mode on
 198:products $MYPRODUCTS_DIR
 199:port-base 1000
 200:<eventlog>
 201:  level info
 202:  <logfile>
 203:    path \$INSTANCE/log/event.log
 204:    level info
 205:  </logfile>
 206:</eventlog>
 207:<logger access>
 208:  level WARN
 209:  <logfile>
 210:    path \$INSTANCE/log/Z2.log
 211:    format %(message)s
 212:  </logfile>
 213:</logger>
 214:<http-server>
 215:  # valid keys are "address" and "force-connection-close"
 216:  address $ZOPE_PORT
 217:  # force-connection-close on
 218:</http-server>
 219:<zodb_db temporary>
 220:    # Temporary storage database (for sessions)
 221:    <temporarystorage>
 222:      name temporary storage for sessioning
 223:    </temporarystorage>
 224:    mount-point /temp_folder
 225:    container-class Products.TemporaryFolder.TemporaryContainer
 226:</zodb_db>
 227:<zodb_db main>
 228:   mount-point /
 229:   # ZODB cache, in number of objects
 230:   cache-size 5000
 231:   <zeoclient>
 232:     server localhost:$ZEO_PORT
 233:     storage 1
 234:     name zeostorage
 235:     var \$INSTANCE/var
 236:     # ZEO client cache, in bytes
 237:     cache-size 20MB
 238:     # Uncomment to have a persistent disk cache
 239:     #client zeo1
 240:   </zeoclient>
 241:</zodb_db>
 242:EOF
 243:
 244:# start the zeo server
 245:$ZEOSERVER_DIR/bin/zeoctl start 
 246:sleep $SLEEPTIME # give the zeo server a chance to start
 247:
 248:# add a Plone Site Instance
 249:cd $base
 250:export PYTHONPATH=$ZOPE_HOME/lib/python:$PYTHONPATH
 251:$PYTHON plonehelper.py -c $ZEOCLIENT0_DIR/etc/zope.conf -i $PloneId -t $PloneTitle -d "$PloneDescription" -u $INSTANCE_USER -p $INSTANCE_PASSWORD  
 252:
 253:# Install our new skin product
 254:$PYTHON plonehelper.py -c $ZEOCLIENT0_DIR/etc/zope.conf -i $PloneId -n $SkinName
 255:
 256:# Set the default skin to our skin
 257:$PYTHON plonehelper.py -c $ZEOCLIENT0_DIR/etc/zope.conf -i $PloneId -s $SkinName
 258:
 259:$ZEOSERVER_DIR/bin/zeoctl stop
 260:
 261:cat << EOF > $base/$StartPloneScript
 262:#!/bin/bash
 263:set -x
 264:$ZEOSERVER_DIR/bin/zeoctl start
 265:sleep $SLEEPTIME # to allow zeo to start
 266:$ZEOCLIENT0_DIR/bin/zopectl start
 267:sleep $SLEEPTIME # to allow client 0 to start
 268:echo "Plone site is running on http://localhost:$ZOPE_PORT/$PloneId"
 269:EOF
 270:
 271:cat << EOF > $base/$StopPloneScript
 272:#!/bin/bash
 273:set -x
 274:$ZEOCLIENT0_DIR/bin/zopectl stop
 275:$ZEOSERVER_DIR/bin/zeoctl stop
 276:EOF
 277:
 278:chmod +x $base/$StartPloneScript
 279:chmod +x $base/$StopPloneScript
 280:
 281:# set up the ipython zope script
 282:cat << EOF > $base/$StartIPythonScript
 283:#!/bin/bash
 284:set -x
 285:export PYTHONPATH=$ZOPE_HOME/lib/python:$PYTHONPATH
 286:$PYTHON -i -c "from Zope import configure;configure('$ZEOCLIENT1_DIR/etc/zope.conf');import Zope; app=Zope.app();ns={'__name__':'blah','app':app};import IPython;IPython.Shell.IPShell(user_ns=ns).mainloop(sys_exit=1);"
 287:EOF
 288:
 289:chmod +x $base/$StartIPythonScript
 290:
 291:# set up a python script to allow debugging from various python debuggers
 292:cat << EOF > $base/$PythonDebugScript
 293:import sys, os, time
 294:print 'starting zeo server'
 295:os.system("$ZEOSERVER_DIR/bin/zeoctl start")
 296:print 'waiting for $SLEEPTIME seconds to allow zeo server to complete startup'
 297:time.sleep($SLEEPTIME)
 298:ZOPE_HOME="$ZOPE_HOME"  
 299:SOFTWARE_HOME= os.path.join(ZOPE_HOME,"lib","python")
 300:sys.path.insert(0, SOFTWARE_HOME )
 301:import Zope , ZPublisher
 302:sys.argv = ['run.py', '-C', '$ZEOCLIENT0_DIR/etc/zope.conf']
 303:Zope.Startup.run.run()
 304:EOF
 305:
 306:echo "Finished, you can either:"
 307:echo "run ./$StartPloneScript to start plone on http://localhost:$ZOPE_PORT/$PloneId"
 308:echo "the run ./$StartIPythonScript to inspect zope/plone api and ZODB"
 309:echo "run ./$StopPloneScript to stop plone"
 310:echo "or run (from your development environment)"
 311:echo "python $PythonDebugScript"
 312:echo "run ./$StopPloneScript to stop the zeo server when you stop the python script"
 313:
plonehelper.py
   1:"""
   2:plonehelper.py -c configFile -i plone_id -t plone_title -d plone_description -u username -p password
   3:
   4:functions to add and adminsiter a Plone Site from script
   5:"""
   6:import sys
   7:import getopt
   8:from Zope.Startup.run import configure
   9:import Zope
  10:import urllib
  11:#from Products.CMFPlone import transaction
  12:import time
  13:
  14:version=0.1
  15:
  16:def add_plone_site(app, plone_id, plone_title, plone_description, username, password):
  17:    req ='/manage_addProduct/CMFPlone/manage_addSite?id=%s&title=%s&create_userfolder=1&description=%s&custom_policy=Default+Plone&submit=+Add+Plone+Site+'%(urllib.quote(plone_id), urllib.quote(plone_title), urllib.quote(plone_description))
  18:    Zope.debug(req,u='%s:%s'%(username, password))
  19:    
  20:def set_default_skin(ploneSite, skin_name):
  21:    # set our new skin as the default skin        
  22:    ploneSite.portal_skins.default_skin = skin_name
  23:        
  24:def install_product(ploneSite, product_name):
  25:    # install the product
  26:    quick_installer =  ploneSite.portal_quickinstaller        
  27:    quick_installer.installProduct(product_name)
  28:    #print  quick_installer.listInstalledProducts()
  29:
  30:def main(argv = None):
  31:    if argv is None:
  32:        argv = sys.argv                
  33:    config_file = None
  34:    username = ''
  35:    password = ''
  36:    plone_id = ''
  37:    plone_title = None
  38:    plone_description = ''    
  39:    skin_name = None
  40:    product_name = None
  41:    try:
  42:        opts, args = getopt.getopt(argv[1:], "hvc:i:t:d:u:p:s:n:", ["help", "version"])
  43:    except getopt.error, msg:
  44:        print msg
  45:        print(__doc__)
  46:        sys.exit(0)        
  47:    for o, a in opts:
  48:        if o in ("-h", "--help"):
  49:            print(__doc__)
  50:            sys.exit(0)
  51:        elif o in ("-v", "--version"):
  52:            log("version: %s"%version)
  53:            sys.exit(0)
  54:        elif o in ("-c"):
  55:            config_file = a
  56:        elif o in ("-u"):
  57:            username = a            
  58:        elif o in ("-p"):
  59:            password = a
  60:        elif o in ("-i"):
  61:            plone_id = a
  62:        elif o in ("-t"):
  63:            plone_title = a
  64:        elif o in ("-d"):
  65:            plone_description = a
  66:        elif o in ("-s"):
  67:            skin_name = a
  68:        elif o in ("-n"):
  69:            product_name = a
  70:    
  71:    if config_file is not None:
  72:        configure(config_file)
  73:        app = Zope.app()
  74:        if plone_id is not None:
  75:            if plone_title is not None:
  76:                add_plone_site(app, plone_id, plone_title, plone_description, username, password)
  77:            else:          
  78:                ploneSite = eval('app.'+plone_id)
  79:                if ploneSite != None:
  80:                    if product_name is not None:
  81:                        install_product(ploneSite, product_name)             
  82:                    if skin_name is not None:
  83:                        set_default_skin(ploneSite, skin_name)
  84:                else:
  85:                    print 'Could not find plone site with id ', plone_id
  86:                # commit these changes        
  87:                get_transaction().commit()
  88:                #transaction.commit()
  89:        else:
  90:            print 'you must specify a unique plone id to create a plone site'
  91:    else:
  92:        print 'you must supply a configuration file: /path/to/zope.conf'
  93:
  94:if __name__=='__main__':
  95:    main()