It's now more then one week since Perl QA Hackathon 2011 in Amsterdam that I've attended. I'll first write about the work part or the official reason why we all went there.

In the beginning every attendee wrote down a plan and then the results at the end after sitting in a meeting room with a laptop and a bunch of hacking chums™ for three days.

My goal in the beginning was to do something related to Debian-Perl. I've asked at the mailing list and got an answer from Gregor that there is an open tasks list and that providing the Debian <-> CPAN mapping would be also useful. I decided for the latter and created a web page with a look-up service:

http://pkg-perl.alioth.debian.org/cpan2deb/

How does it works? The page is a result of three scripts - dpkg-scanpmpackages, apt-pm and apt-pm-web and the Debian::Apt::PM module behind them. While only apt-pm-web was not existing before the hackathon the module also needed some love in order to be able to get the distribution name and the component information in the output too. Here is how the scripts works:

dpkg-scanpmpackages /path/to/repository-or-deb-mirror

will find all Packages.bz2 files in the given path, parse them, extract the .deb package files, look-up for all .pm files inside, analyse them for Perl package names and write all this informtion to the same folder into the PerlPackages.bz2 file. (here's example) Each .deb file has an own section in this file like:

Package: libcontextual-return-perl
Architecture: all
Filename: pool/main/libc/libcontextual-return-perl/libcontextual-return-perl_0.003001-1_all.deb
Version: 0.003001-1
Distribution: stable
Component: main
Perl-Modules: 
 Contextual::Return::Failure (0)
 Contextual::Return (0.003001)

apt-pm update

will scan all repositories (all components) listed in /etc/apt/sources.list and fetch the PerlPackages.bz2 files. There are special empty package repositories with only PerlPackages created that can be safely added to the sources.list:

deb http://pkg-perl.alioth.debian.org/~jozef-guest/pmindex/     lenny   main contrib non-free
deb http://pkg-perl.alioth.debian.org/~jozef-guest/pmindex/     squeeze main contrib non-free
deb http://pkg-perl.alioth.debian.org/~jozef-guest/pmindex/     wheezy       main contrib non-free
deb http://pkg-perl.alioth.debian.org/~jozef-guest/pmindex/     sid          main contrib non-free
deb http://pkg-perl.alioth.debian.org/~jozef-guest/pmindex/     experimental main contrib non-free

or even local file:// non-public repository will work fine. Then using `apt-pm find Module::Name` a module can be looked-up, if it is already present in any of the specified distributions/components.

apt-pm-web

will loop through all the modules listed in 02packages.details.txt.gz (on CPAN) and generate index .json files to be used on the web. These .json files with all the rest of the web files are then copied to alioth. These page is basically a page with single ajax form input field. The dynamic part is written in JavaScript in script.js. Once the form is submitted, the value is md5-ed, first two letters are used for the name of the json file which is loaded and if the module name is present the results are show on the page. This way there is no need for server-side code to run and all that alioth (webserver) has to do is to shoot static files. Also note the the url looks like this http://pkg-perl.alioth.debian.org/cpan2deb/#q=Tk::Widget where the "q" (query) parameter is delimited via "#" instead of normal "?" resulting in no page refresh when changing the url or resubmitting the form.

So what's there for you?

Now there is an easy way how to look-up Perl modules in Debian packages on the web. And there is a way to do this for a local non-public repositories too. In case you have local repository that includes packaged Perl modules and would like to have this page locally, here is a how-to guide:

1. cpan -i Debian::Apt::PM
2. editor /etc/apt/sources.list # add one/some of the above repositories and your local one
3. dpkg-scanpmpackages /path/to/local-repository
4. apt-pm update
5. apt-pm find DBI # to test if all works fine
6. git clone https://github.com/jozef/Debian-Apt-PM.git /tmp/dapm
7. cp -r /tmp/dapm/examples/web /srv/cpan2deb
8. perl -MCPAN -MCPAN::Shell -le 'CPAN::Shell->reload("index")'
9. apt-pm-web -p $HOME/.cpan/sources/modules/02packages.details.txt.gz -o /srv/cpan2deb'
10. # make the /srv/cpan2deb folder available via your favourite web server

Actually people from Ubuntu community could find use for this and create such a page for the many Ubuntu distributions that are out there. (I can not do this as I don't have a disk space and CPU to process such a huge mirror. Making it work without the need for all the .deb files that could be deleted after indexing would help, but I don't have the time to do this neither)

To be continued...