My goal was to have a single command to use to install Perl modules that will prefer distribution packages over installing from CPAN.

Combining 02packages.details.txt.gz with data from MetaCPAN::API I've generated 02packages.dependencies.txt.gz. This file has similar structure, basically it is a list of module names with all the dependencies it has. Here are example lines:

DBI     1.616   Clone/0.31 DBD::AnyData/0.09 DBD::CSV/0.29 DBD::PO/2.10 DBD::RAM/0.072 DB_File ExtUtils::MakeMaker ExtUtils::MakeMaker/6.48 MLDBM Net::Daemon RPC::PlServer/0.2001 SQL::Statement/1.27 SQL::Statement/1.28 Test::Simple/0.90 perl/5.008
Moose   2.0205  Algorithm::C3 DBM::Deep Data::OptList/0.107 Data::Visitor DateTime DateTime::Calendar::Mayan DateTime::Format::MySQL Declare::Constraints::Simple Devel::GlobalDestruction Devel::PartialDump/0.14 Dist::CheckConflicts/0.02 Eval::Closure/0.04 ExtUtils::MakeMaker/6.30 File::Find::Rule HTTP::Headers IO::File IO::String List::MoreUtils/0.28 Locale::US MRO::Compat/0.05 Module::Info Module::Refresh Package::DeprecationManager/0.11 Package::Stash/0.21 Package::Stash::XS/0.18 PadWalker Params::Coerce Params::Util/1.00 Regexp::Common Scalar::Util/1.19 Sub::Exporter/0.980 Sub::Name/0.05 Task::Weaken Test::Deep Test::DependentModules/0.09 Test::Fatal/0.001 Test::Inline Test::LeakTrace Test::More/0.88 Test::Output Test::Requires/0.05 Try::Tiny/0.02 URI perl/v5.8.3
MetaCPAN::API   0.33    Any::Moose Carp ExtUtils::MakeMaker/6.30 File::Find File::Temp HTTP::Tiny JSON Module::Build/0.3601 Test::Fatal Test::More Test::TinyMocker Try::Tiny URI::Escape
Pod::POM::Web   1.17    Alien::GvaScript/1.021 AnnoCPAN::Perldoc::Filter Config Encode::Guess HTTP::Daemon List::MoreUtils List::Util MIME::Types Module::Build/0.38 Module::CoreList POSIX PPI::HTML Pod::POM/0.25 Pod::POM::View::HTML Search::Indexer/0.75 Test::More Time::HiRes URI URI::QueryParam
Pod::POM::Web::Indexer  0       Alien::GvaScript/1.021 AnnoCPAN::Perldoc::Filter Config Encode::Guess HTTP::Daemon List::MoreUtils List::Util MIME::Types Module::Build/0.38 Module::CoreList POSIX PPI::HTML Pod::POM/0.25 Pod::POM::View::HTML Search::Indexer/0.75 Test::More Time::HiRes URI URI::QueryParam
Pod::POM::Web::PSGI     0.002   CGI::Emulate::PSGI/0.12 ExtUtils::MakeMaker/6.30 Pod::POM::Web/1.17

So basically it's possible to download this file and find out offline the dependency chain of a module.

Which I did in apt-cpan. With help of Debian::Apt::PM if is possible to preferably install packages from Debian distribution repositories and only the missing modules via CPAN shell.

Here is an example:

$ apt-cpan -n install Nagios::Plugin::OverHTTP
sudo apt-get install libdata-validate-domain-perl libdata-validate-uri-perl libenv-path-perl libhtml-strip-perl libmoosex-clone-perl
sudo cpan -i Nagios::Plugin::OverHTTP Const::Fast

Slides from my yesterdays talk are here, uploaded to Bratislava.pm.org page.

My thanks go to everyone who participated on this great conference and helped this event happen!

Let this blog post be also an invitation to my talk that I'll have on this years YAPC::EU::2011 in Riga and a reason "Why?" there will be more questions then answers.

There is one question that keeps me busy for quite some months now already and that is: What if the knowledge and wisdom is not in answers, but in questions?

As it kept me busy for just too long and as I'm a full-time Perl programmer, I've decided to write a code to help me answer "Why?".

package Acme::KnowledgeWisdom;

our $VERSION = '0.01';

use Moose;
use warnings FATAL => 'all';

has 'in_questions' => ( isa => 'Bool', is => 'ro', default => 1);
has 'has_already'  => ( isa => 'Bool', is => 'ro', default => 0);

sub get {
    my $kw = shift;
    
    return $kw->ask
        if $kw->in_questions;

    return 42;
}

sub ask {
    my $self = shift;
    
    return 42
        if $self->has_already;
    
    my $kw = Acme::KnowledgeWisdom->new;
    return $kw->get;
}

1;

The module is well tested ;-) and is of course on CPAN.

After the theory, first wrong solution, here comes the second wrong solution. It's easier one and the JavaScript is used only for iPhone-s where it was also tested. The problem with iPhone 3 and 4 is that both hardware versions are using the same OS and because of that the same user-agent string. So there is no way of knowing which resolution does the device have when the HTTP request comes to the server.

Following two meta viewport tags works fine for all phones besides the iPhone. The first one is when scaling should be disabled, the other when enabled.

<meta name="viewport" content="width=device-width, minimum-scale=0.5, maximum-scale=1.0, user-scalable=no, target-densitydpi=device-dpi"/>
<meta name="viewport" content="width=device-width, minimum-scale=0.5, maximum-scale=3.0, user-scalable=yes, target-densitydpi=device-dpi"/>

To get the native resolution for the iPhone 4 I had to create a JavaScript that detects 320px vs 640px resolution, sets the cookie and makes a refresh while setting the url parameter so that it also works when cookies are disabled. Then our server-side code has to resize the images properly and update the "width=640" in the viewport meta tag. Still a little complex, right? But works! :-)

Now what are the problems.The fonts can get pretty small as iPhone 4 applies the 0.5 scale and the Androids with high resolution uses the high-densitydpi. And it doesn't work so well for Opera too.

Here is the JavaScript code (much less lines then the previous one):

The part1 described some troubles with the screen sizes of the mobile devices. Since then I spend some time playing around with the problem and I found 2 wrong solutions until I got the (hopefully) right one. So let's start with the first of the wrong ones.

The solution relies on the fact that we have the device screen width and height in our DMS (Device-Management-System) database and this is compared with the real width of the device using JavaScript, then if there is a difference a cookie is set and via this cookie our Renedering-Engine on the web server is instructed to resize the images using the zoom level. It requires refresh, lot of "junk" in the HTML header and all the "machinery" on the server side. But the images are properly fit even for Opera browser that has the same user-agent string for different devices with different screen sizes.

See the code below, it's a nice example of a working-complex-bad solution :-)

It's now more then one week since Perl QA Hackathon 2011 in Amsterdam that I've attended. I'll first write about the work part or the official reason why we all went there.

In the beginning every attendee wrote down a plan and then the results at the end after sitting in a meeting room with a laptop and a bunch of hacking chums™ for three days.

My goal in the beginning was to do something related to Debian-Perl. I've asked at the mailing list and got an answer from Gregor that there is an open tasks list and that providing the Debian <-> CPAN mapping would be also useful. I decided for the latter and created a web page with a look-up service:

http://pkg-perl.alioth.debian.org/cpan2deb/

How does it works? The page is a result of three scripts - dpkg-scanpmpackages, apt-pm and apt-pm-web and the Debian::Apt::PM module behind them. While only apt-pm-web was not existing before the hackathon the module also needed some love in order to be able to get the distribution name and the component information in the output too. Here is how the scripts works:

dpkg-scanpmpackages /path/to/repository-or-deb-mirror

will find all Packages.bz2 files in the given path, parse them, extract the .deb package files, look-up for all .pm files inside, analyse them for Perl package names and write all this informtion to the same folder into the PerlPackages.bz2 file. (here's example) Each .deb file has an own section in this file like:

Package: libcontextual-return-perl
Architecture: all
Filename: pool/main/libc/libcontextual-return-perl/libcontextual-return-perl_0.003001-1_all.deb
Version: 0.003001-1
Distribution: stable
Component: main
Perl-Modules: 
 Contextual::Return::Failure (0)
 Contextual::Return (0.003001)

apt-pm update

will scan all repositories (all components) listed in /etc/apt/sources.list and fetch the PerlPackages.bz2 files. There are special empty package repositories with only PerlPackages created that can be safely added to the sources.list:

deb http://pkg-perl.alioth.debian.org/~jozef-guest/pmindex/     lenny   main contrib non-free
deb http://pkg-perl.alioth.debian.org/~jozef-guest/pmindex/     squeeze main contrib non-free
deb http://pkg-perl.alioth.debian.org/~jozef-guest/pmindex/     wheezy       main contrib non-free
deb http://pkg-perl.alioth.debian.org/~jozef-guest/pmindex/     sid          main contrib non-free
deb http://pkg-perl.alioth.debian.org/~jozef-guest/pmindex/     experimental main contrib non-free

or even local file:// non-public repository will work fine. Then using `apt-pm find Module::Name` a module can be looked-up, if it is already present in any of the specified distributions/components.

apt-pm-web

will loop through all the modules listed in 02packages.details.txt.gz (on CPAN) and generate index .json files to be used on the web. These .json files with all the rest of the web files are then copied to alioth. These page is basically a page with single ajax form input field. The dynamic part is written in JavaScript in script.js. Once the form is submitted, the value is md5-ed, first two letters are used for the name of the json file which is loaded and if the module name is present the results are show on the page. This way there is no need for server-side code to run and all that alioth (webserver) has to do is to shoot static files. Also note the the url looks like this http://pkg-perl.alioth.debian.org/cpan2deb/#q=Tk::Widget where the "q" (query) parameter is delimited via "#" instead of normal "?" resulting in no page refresh when changing the url or resubmitting the form.

So what's there for you?

Now there is an easy way how to look-up Perl modules in Debian packages on the web. And there is a way to do this for a local non-public repositories too. In case you have local repository that includes packaged Perl modules and would like to have this page locally, here is a how-to guide:

1. cpan -i Debian::Apt::PM
2. editor /etc/apt/sources.list # add one/some of the above repositories and your local one
3. dpkg-scanpmpackages /path/to/local-repository
4. apt-pm update
5. apt-pm find DBI # to test if all works fine
6. git clone https://github.com/jozef/Debian-Apt-PM.git /tmp/dapm
7. cp -r /tmp/dapm/examples/web /srv/cpan2deb
8. perl -MCPAN -MCPAN::Shell -le 'CPAN::Shell->reload("index")'
9. apt-pm-web -p $HOME/.cpan/sources/modules/02packages.details.txt.gz -o /srv/cpan2deb'
10. # make the /srv/cpan2deb folder available via your favourite web server

Actually people from Ubuntu community could find use for this and create such a page for the many Ubuntu distributions that are out there. (I can not do this as I don't have a disk space and CPU to process such a huge mirror. Making it work without the need for all the .deb files that could be deleted after indexing would help, but I don't have the time to do this neither)

To be continued...

I spend some time recently fiddling around with meta viewport, so I'll share some notes. First here are the specifications:

• Apple
• Google
• Mozilla
• Nokia (but they are referring to an Apple documentation)
• QtWebKit
• Microsoft

The blog posts A pixel is not a pixel is not a pixel and The two viewports are good to read describing the "problem" with zooming and viewport. The trouble is that high dpi mobile devices are zooming in by default. Speciality of the iPhone is that the user-agent string is totally the same for 320px iPhone 3 as for the iPhone 4 which is 640px wide. To get the nice sharp images on an iPhone 4 one needs to use 640px images and set the image width attribute to 100%. At my $work we resize images to exact size of the mobile devices. With this auto-zooming features of modern devices the life's getting more difficult.

And here is what I found regarding the <meta name="viewport"> tag. This are two+1 generally usable examples:

<meta name="viewport" content="width=device-width, minimum-scale=0.5, maximum-scale=1.0,
    user-scalable=no, target-densitydpi=device-dpi"/>
<meta name="viewport" content="width=device-width, minimum-scale=0.5, maximum-scale=3.0,
    user-scalable=yes, target-densitydpi=device-dpi"/>
<meta name="viewport" content="width=640,          minimum-scale=0.5, maximum-scale=3.0,
    user-scalable=yes, target-densitydpi=device-dpi"/>

• width=device-width will tell the browsers to use the native resolution of the phone.
• target-densitydpi=device-dpi is needed for the phones with Android and a high-resolution (or high-dpi, example screens with 480x800, like Samsung Galaxy S) to not zoom. Note that the target-density is at the end of the content attribute. This is because it is unrecognised by iPhone and if iPhone encounters and unrecognised option in the viewport content attribute it stops parsing and will ignore the rest of the string.
• maximum-scale=1.0 together with user-scalable=no is there to forbid zooming in. If the page is optimized for the mobile resolution it's ok to disable it.
• maximum-scale=3.0 together with user-scalable=yes can be used to let the freedom to the users to make the fonts/links/images bigger. Just one note here. Once the user applies his own zoom on the page there is no way with JavaScript to get the native screen resolution.
• minimum-scale=0.5 is a little bit tricky. The 0.5 is there for the iPhone 4. This device under normal conditions is rendering images 320px wide to the full screen width, which means zooming them in => 320px images is shown on the full width of 640px wide screen. I found just one way to force iPhone 4 to show 640px images natively (without width attribute) and that is to set width=640 in the viewport meta tag. The problem is that the only way to recognize iPhone 3 from iPhone 4 is via JavaScript and window.devicePixelRatio == 2. But the JavaScript is a little bit too late for the content of the page.
• width=640 setting the proper (fixed) width is also necessary for the windows mobile phones with IE in order to have the native resolution.

This is not all. There are device databases (WURFL, DeviceAtlas) that collect user-agent strings and helps to provide screen widths to the mobile page rendering engines, but for an Opera Mini & Opera Mobile the user-agent string is the same/similar for all kinds of devices. And the change of device orientation is also worth to consider, as just by turning the phone around, width of the screen changes from 480px to 800px.The only way to get the screen resolution is in JavaScript, but that is too late, the page is already there...

One solution I've described in the beginning and that is to let the browser scale down the high-resolution pictures. This works for the modern phones and is not too bandwidth friendly (providing 640px wide images to all types of phones) and not so reliable with the old phones either. Another solution, that I'm testing currently, is to try to guess the screen width as good as possible based on user-agent string and then use JavaScript to get the proper screen width, reloading the page with a cookie set if necessary (if the width is not what was expected) or having the proper width set for the next page request.

One final thought - Why don't the mobile browsers send the display width and height in the http headers? This will save us a lot of troubles with device detection.

I wanted to get some CPAN authors sample. Not everyone as there are too much (> 8000). So I've created this two on-liners to get the CPAN authors whose module is packaged for Debian:

apt-cache search perl | perl -lane 'next if $_ !~ m/^lib(.+?)-perl\s/; $m=$1; $m=~s/-/::/g; print $m;' > /tmp/deb-perl.txt

zcat ~/.cpan/sources/modules/02packages.details.txt.gz | perl -MIO::Any -lae 'my %x; BEGIN { %x=map { $_ => 1 } split "\n", IO::Any->slurp("/tmp/deb-perl.txt") }; while (<>) { @F=split /\s+/; $m = lc $F[0]; if ($x{$m}) { (undef,undef,$a)=split "/", $F[2]; print $a; }; }' | sort | uniq | xargs echo

And here is the result (842 out of 8270 total):

Here are some variants that I've seen so far.

1. Careful people that are afraid to spoil the system use:

   • /usr/local
   • /var/local
   • /opt

2. Crazy people have always an original place to hide files in:

   • /usr/local64
   • /usr/local/64
   • /usr/local/local
   • /shared
   • /corp
   • /data
   • ...

3. Linux distributions put files according to the Filesystem Hierarchy Standard.

Some years ago I belonged to the first group and I was putting everything into /usr/local to be sure that there is just my stuff. Today I belong to the second one... I use /data for my laptop. Which is not so great idea, but it is my laptop and no one ever will have to (won't even be able) to touch it, so it is my mess. Some of the other ones like /corp, /shared and /usr/local64 I have to use at $work, because it is ops decision or a historical reason (sniff, sniff).

I hope one day I'll further evolve to work according to the standards. Tiding-up my mess would be the easy part of the journey...

Some people say that specifications are useless. That they never reflect the reality, that by the time of being done they are out-dated and people should just rush out, start coding and start working. I found this nice quote:

That was my missing piece to the argumentation. May be the specs at the end are "not so accurate", but at the time of writing them, the idea has to be exercised, questioned, the missing gaps filled in, all details understood and the goal clearly defined.