The file structure of my project

At the moment the directory structure of my project built upon Sphinx’s dir structure looks like this:

builders/webapp/ -> webapp builder used by Sphinx to build the docs and the webapp,
builders/webapp/templates/ -> the *.py template files rendered by webapp builder (at the moment one file – server.py – lies here, but actually this file is simply copied, not rendered, but it might change in the future),
builders/webapp/templates/html/ -> the templates of HTML code rendered by the middleware (imported by server.py) when serving the docs,

web/middleware/ -> middleware for Xapian, OpenID and serving html files in general,

themes/basic/web/ -> templates for full HTML files. These files are used as templates twice. First, when Sphinx is building the docs, it extends these files with “layout.html”, and then, for a second time, when the files are accessed through the browser. That’s why these files are separated from those located in builders/webapp/templates/html/ directory, which are rendered only once, by the middleware.

A bit of mess, isn’t it? Comments are appreciated 🙂

The structure for generated docs and webapp looks like this (to be found in _build/webapp):

html/ -> files from builders/webapp/templates/html and themes/basic/web directories are located here. We don’t have to separate them anymore, because they are all used as templates by middleware now.

openidstore/ -> OpenID directory. It is created and administered by python-openid library.

public/ -> files built by Sphinx go here. This is the directory which should be served

xapian_db/ -> Xapian’s database.

server.py -> a script which imports the middleware and serves the docs. The idea is that: by adapting the code from this file to one’s framework, one should be able to use this webapp in his python web framework of choice. I plan to change it soon, so all server.py logic will be available as a middleware, because the middleware (Xapian, OpenID, etc.) pieces are actually unseparable, so there is no need for an ordinary user to care about importing them all.
Nevertheless, that’s how it looks like now. It may all change in the future, but if we are about to discuss it, it’s good to describe it as it is first.

Next step: OpenID logging

OK, it is now possible to login with one’s OpenID login after the docs and webapp for serving the documentation were generated. It took me some time to make it work, mainly because Paste’s module for OpenID logging – paste.auth.open_id – is rather out of date.

What I needed to do to make it work was to put together the WSGI architecture and the OpenID logging logic. The examples from python-openid package were very helpful, though. It is working already, but there are some problems I still need to resolve. To name a few: add a way to distinguish admins from ordinary users (anyone can log in, but only choosen users have special rights), add a “logged in as…” line to the templates (the problem here is that the docs are used as templates twice: when Sphinx generates the docs and later on when webapp is displaying them).

Nevertheless, you can try it just as described in earlier posts. Also, until June 22 a form for posting comments should be ready. Seems like busy weekend! 🙂

Integration of Xapian

It already works. I still can point to some problems, but the Xapian search engine is quite usable and it acknowledges themes choosen by the user.

A “search” form for Xapian is placed under a traditional “quick search” form. You can already build your documentation with this addition. Simple “make webapp” makes the job.

New approach: WSGI+WebOb instead of Pylons

Quite big change in my plans. I dropped strict support of Pylons. The community is interested in running the webapp on different frameworks and thus I would like to ease their lives as much as possible. My aim is to stay PEP-333 compliant and to build pure WSGI+WebOB code.

The procedure of building the webapp from Sphinx is as it was before (make webapp), but now it is not possible to run pylons to serve the docs, but a simple file serving software is provided.

After building a webapp the directory structure looks like this:

$ ls _build/webapp/
html/
public/
server.py
xapian_db/

The “html” directory contains templates for webapp. The documentation generated by Sphinx is located in “public” directory. “server.py” serves the documentation (127.0.0.1:8000 by default). “xapian_db”, as the name says, is a directory where Xapian’s search engine database is located.

Xapian will be described in the next post, though!

Integrating Pylons into Sphinx

Sphinx is now able to build a webapp and serve the documentation. Let’s take a look at how it works, step by step (I assume that you got Sphinx from my repo on Bitbucket installed):

1. Start a new Sphinx project:

$ mkdir sphinx-project
$ cd sphinx-project/
$ sphinx-quickstart
[…usual sphinx-quickstart questions…]

2. Our documentation is empty as for now, but still we can build a webapp which will serve the default documentation template:

$ make webapp
[…]
Build finished. The webapp HTML pages are in _build/webapp.

3. Run the server:

$ cd _build/webapp/WebAppProject/
$ paster serve –reload development.ini
Starting subprocess with file monitor
Starting server in PID 6119.
serving on http://127.0.0.1:5000

OK, so there it goes!

Let’s take a look:

sphinx-empty

The usual way of starting a Pylons project is (in my project I import CreateDistroCommand class from paste.script.create_distro instead of executing paster):

$ paster create –template=pylons TemplateDemo

Paster then asks two questions to the user:

Enter template_engine (mako/genshi/jinja2/etc: Template language) [‘mako’]:
Enter sqlalchemy (True/False: Include SQLAlchemy 0.5 configuration) [False]:

So, by running the paster command we are passing 4 arguments: the template name (“pylons”), the project directory (“TemplateDemo”), the template_engine (by default: “mako”) and sqlalchemy (by default: False). Since it would be a waste of time to answer those questions anytime you do a rebuild (“make clean; make webapp”), these values are configureable through Sphinx’s conf.py file:

paster_template = ‘pylons’
paster_sqlalchemy = False
paster_template_engine = ‘jinja2’
paster_project_name = ‘WebAppProject

Set them once, and set them for good.

OK, the basic integration of Pylons and Sphinx is done. Now it’s time to work on Pylons webapp!

Let’s begin!

GSoC is up and running for two days now. Unfortunately, just as I thought, the end of May is occupied by the Uni assignments and thus, the coding must wait.

Nevertheless, in the meantime I have created my GSoC repository on BitBucket. You can view it in here or clone it for your own needs in this way:

$ hg clone http://bitbucket.org/gminick/sphinx-gsoc2009/

Cheers, and good luck to fellow GSoCers.

Yet another new and shiny blog

Welcome, this is my Google Summer of Code 2009 blog. Its main purpose is to share my weekly progress reports with you through June, July and August.

You can read my application in here.

For more info about me, check this out.