Archive for the ‘Perl’ Category

Monthly Donations: Perl Dancer and the “Friends to the Animals” Foundation

Monday, June 30th, 2014

I’m blogging from the scene of destruction and havoc that used to be my apartment, but it’s currently undergoing renovation and this is what it’s turned into. There is dust everywhere, the furniture is all over the place, and I’m having trouble finding my way around. In short, it’s like a major rewrite of a large code base, but in real life.

Anyway, some renovation-shmenovation won’t stop me from continuing the initiative of monthly donations — so here we go. It has become a tradition of its own that June is the month when I support Dancer, my favorite Perl web framework, which I use both for work and for my personal projects with great joy. This year is no exception, and with my donation to the project I’m sending the well-deserved big thanks to the developers and maintainers of the project.

The second donation that I’m making this month goes to the “Friends to the Animals” Foundation, based in Katowice (whom I also support regularly — the previous time was in November last year). Their mission is to help mistreated and abandoned animals and to raise awareness of animal welfare issues, and I know they are truly dedicated to their cause. Thank you for your great work, Friends to the Animals!

Dancer::Plugin::Preprocess::Markdown

Monday, November 25th, 2013

Last month, while working on the POD Web View site, I developed a small Dancer plugin that lets you easily convert Markdown files into HTML content in your web application — Dancer::Plugin::Preprocess::Markdown.

The generated HTML content can be saved in a file, so that the Markdown source is only processed when it gets changed. Thus, the plugin can be used to build a poor man’s substitute for a static site generator (like Jekyll).

You’ll find the code at the usual places: CPAN and GitHub. As always, I’m looking forward to feedback/patches/forks.

POD Web View

Wednesday, October 23rd, 2013

When developing a Perl module, I often want to get a quick preview of the documentation that I’m writing, just to see if everything is in order and how it turns out. I used to do this the old fashioned way, by generating an HTML file with pod2html or pod2cpanhtml and opening it in a browser, but I was hoping in this day and age there is an easier and better solution, preferably a web application.

Looking around, however, the only thing I could find was the pod2html page at the CPAN Search site, which allows you to upload a POD file, have it processed by pod2html, and displayed with CPAN style. I thought it might be a good idea to try building something more user-friendly, with features like editing POD in the browser, drag and drop file uploads, etc.

And what better time for a little project like this than a weekend when you’re ill and not supposed to leave your apartment? Well, that’s what my last weekend was like — two days of coughing and coding, and here’s the result: POD Web View.

The application allows you to upload a POD file, get it from a URL, or paste its contents and edit it on the fly. The generated HTML can be displayed in the style of your choice, mimicking how it would look on CPAN, MetaCPAN, or GitHub.

To give credit where it’s due, the backend is built on Dancer and uses Pod::Simple::HTML to generate the HTML preview. The user interface is made with Twitter Bootstrap, a lot of JavaScript/jQuery code, and the amazing Ace editor.

I hope this will be useful for at least a few fellow Perl developers, like it already is for me. Please note that at this point this is still work in progress — the backend code needs some more work (e.g. basic sanity checks), and there are a couple UI issues that I’m aware of (and likely a dozen more that I’m not). Anyway, be my guest and give it a try, and if you’d like to report an issue, or maybe help me with the development (more than welcome), I’ve put the project up on GitHub.

HTTP::Tiny::SPDY – A Subclass of HTTP::Tiny with SPDY Support

Tuesday, October 15th, 2013

I have recently returned to working on Arriba, the PSGI-compliant web server with support for the SPDY protocol, modeled on Starman.

I sort of released it (on GitHub only, not on CPAN) back in January, at that time the code was passing the tests in Plack::Test::Suite when running as a regular HTTP/HTTPS server. My next goal, before considering the module ready to be released on CPAN, was to make it pass those tests in SPDY mode. This meant I needed to add support for SPDY to good old LWP::UserAgent, which was used as the HTTP client in Plack tests.

Over the weeks/months that followed, I made a few attemps at tackling this problem, but had a hard time wrapping my head around the architecture of LWP::UserAgent and figuring out a reasonable way to add SPDY into the mix. Having very little time to devote to this project, I didn’t get anywhere with it.

A few days ago, I was delighted to find out that the problem went away by itself, since Plack switched from using LWP::UserAgent to its own Plack::LWPish, which is built around HTTP::Tiny. Now I needed to implement SPDY in HTTP::Tiny, which is, well, tiny when compared to LWP::UserAgent, so the task seemed much easier. I gave it a shot this weekend and got it working in a matter of hours, spawning HTTP::Tiny::SPDY, a subclass of HTTP::Tiny that works the same as the original, but can also do SPDY.

I immediately used the module for the intended purpose of testing Arriba in SPDY mode, and, as expected, this revealed many problems, but most of them turned out to be easy to fix (except for one, which took me more than three hours just because I didn’t RTFM in the first place — will I ever learn?). Soon, Arriba running SPDY was passing all the tests in the suite, which I happily celebrated with a tasty porter beer. I am now cleaning up the code to prepare it to be finally released on CPAN.

And speaking of CPAN, HTTP::Tiny::SPDY is already there, as well as on GitHub. Like Arriba, this is an early release, the code is hackish and immature, and I take zero responsibility for the pain and suffering that you may bring upon yourself when you try to use it. But if you do, I crave your feedback.

Monthly Donations: Perl Dancer and the Treatment of Czajka

Friday, June 28th, 2013

It’s donations time, everybody!

Today I’m once again supporting an open source project that I’ve already donated to last year. There are many projects that I use on a regular basis or even every day, so I feel they deserve more than just a one-time donation. One of these projects is Dancer, the Perl web framework that I admire and which has been powering this site for more than two years now. Sadly, these days I don’t have as much time to contribute to the project as I had in the past (when I wrote a few Dancer plugins), so at least I’m going to help the project with my little donation.

The second of this month’s donations is to help fund the treatment of Czajka, the horse that was rescued from being sold to a slaughterhouse — as you might recall, I wrote about her last month. Medical tests revealed that Czajka suffers from a serious injury to one of her back legs which must have happened sometime in the past. She needs to undergo an operation and have the leg put in a cast, then stay in a horse hospital for two months, until the cast is ready to be removed. As you might imagine, all this costs a lot, so the Gift of Heart Foundation (who is taking care of Czajka) is accepting donations to fund it.

I hope the full amount gets raised soon and Czajka won’t have to wait much longer for her treatment to begin. If any of you, my dear readers, would like to chip in but don’t know how to proceed (because of, for instance, not understanding the Polish website of the Foundation), feel free to contact me and I’ll assist you.

Montly Donations: The Perl Foundation, “Przystan Ocalenie”, and Czajka

Friday, May 31st, 2013

This month is the first anniversary of my monthly donations to open source software and charity organizations, as I started this initiative in May last year. Hooray!

To celebrate the anniversary, I made donations to the same organizations as I did back then, and that is to the Perl Foundation and to “Przystan Ocalenie”, the animal rescue centre.

I also made one additional donation this month — my friend Jakub told me one day that he and his workmates were collecting money to rescue an injured horse destined for slaughterhouse. I chipped in, we collected more than enough and the two-year old filly Czajka has been saved. She’s now undergoing medical tests and treatment in a horse shelter. Here’s a photo of her:

Get better soon, Czajka!

Arriba – PSGI Web Server with SPDY Support

Tuesday, January 8th, 2013

Having some free time during the Christmas break, I decided to give a shot at a small project that I had in mind for a while, and that was to develop a PSGI-compliant(-ish) web server with support for the SPDY protocol.

In case you haven’t heard of it, SPDY is a networking protocol developed at Google with a goal of reducing web page load latency. It is currently used by some of Google services (including search and Gmail) and by Twitter, and is supported natively in Firefox, Chrome, and Opera — so if you visited any of those sites with any of those browsers, it’s highly likely that your web content was transmitted by means of SPDY. An official standard for the protocol is in the works.

There was a SPDY module on CPAN that looked promising — Net::SPDY by Lubomir Rintel. While not being a complete implementation of the protocol, it seemed to be working, as I found out by playing with the sample client and server scripts included in the distribution.

After a few days of reading the SPDY specs, minor reverse engineering of other implementations, and blatantly copying (a lot of) code from Starman, I was able to put together a preforking web server operational enough to run a few simple Dancer applications. It’s a mess and nowhere near being ready for production use, but I’m happy to share it to maybe get some feedback from you fine folks — I’ve put it up on GitHub. I intend to continue working on it and hopefully one day turn it into something half-decent.

If you want to run it, be aware that you currently need to use the Net::SPDY module from my forked repository instead of the original one, since in the original there’s some test code that breaks normal server communication.

About the project name — I followed the idea of using friendly names like Starman and Twiggy, and since SPDY reminds me of Speedy Gonzales, I used a part of Speedy’s catch phrase (“¡Ándele! ¡Ándele! ¡Arriba! ¡Arriba!”). However, I know I’m terrible at naming things, so I’m open to suggestions for a better name.

Happy Birthday Perl

Tuesday, December 18th, 2012

Happy 25th Birthday, Perl! I made you a card:

perl -C -e'print map+(v9635 x8^unpack B8).(v10)[++$n%9],(unpack
    u,"MK__^=^WW_S_WK)FNO\$3F5U\$7BJJ.=>U51S-WK)GNM>UF=W%[K[N>?___SW__")=~/./g'

Building a Search Web App with Dancer and Sphinx

Friday, December 14th, 2012

In this article, we’ll develop a basic search application using Dancer and Sphinx. Sphinx is an open source search engine that’s fairly easy to use, but powerful enough to be deployed in high-traffic sites, such as Craigslist and Dailymotion.

In keeping with this year’s Dancer Advent Calendar trend, the example app will be built on Dancer 2, but it should work just as well with Dancer 1.

Alright, let’s get to work.

The Data

Our web application will be used to search through documents stored in a MySQL database. We’ll use a simple table with the following structure:

CREATE TABLE documents (
    id int NOT NULL AUTO_INCREMENT,
    title varchar(200) NOT NULL,
    contents_text text NOT NULL,
    contents_html text NOT NULL,
    PRIMARY KEY (id)
);

Each document has an unique ID, a title, and contents, stored as both plain text and as HTML. We need the two formats for different purposes — HTML will be used to display the document in the browser, while plain text will be fed to the indexing mechanism of the search engine (because we do not want to index the HTML tags, obviously).

We can populate the database with any kind of document data — for my test version, I used a simple script to fill the database with POD documentation extracted from Dancer distribution. The script is included at the end of this article, in case you’d like to use it yourself.

Installation and Configuration of Sphinx

Sphinx can be installed pretty easily, using one of the pre-compiled .rpm or .deb packages, or the source tarball. These are available at the download page at SphinxSearch.com — grab the one that suits you and follow the installation instructions.

When Sphinx is installed, it needs to be configured before we can play with it. Its main configuration file is usually located at /etc/sphinx/sphinx.conf. For our purposes, a very basic setup will do — we’ll put the following in the sphinx.conf file:

source documents
{
    type        = mysql
    sql_host    = localhost
    sql_user    = user
    sql_pass    = hunter1
    sql_db      = docs

    sql_query   = \
        SELECT id, title, contents_text FROM documents
}

index test
{
    source          = documents
    charset_type    = utf-8
    path            = /usr/local/sphinx/data/test
}

This defines one source, which is what Sphinx uses to gather data, and one index, which will be created by processing the collected data and will then be queried when we perform the searches. In our case, the source is the documents database that we just created. Thesql_query directive defines the SELECT query that Sphinx will use to pull the data, and it includes all the fields from the documents table, except contents_html — like we said, HTML is not supposed to be indexed.

That’s all that we need to start using Sphinx. After we make sure the searchd daemon is running, we can proceed with indexing the data. We call indexer with the name of the index:

$ indexer test

It should spit out some information about the indexing operation, and when it’s done, we can do our first search:

$ search "plugin"

index 'test': query 'plugin ': returned 8 matches of 8 total in 0.002 sec

displaying matches:
1. document=19, weight=2713
2. document=44, weight=2694
3. document=20, weight=1713
4. document=2, weight=1672
5. document=1, weight=1640
6. document=13, weight=1640
7. document=27, weight=1601
8. document=28, weight=1601

Apparently, there are 8 documents in the Dancer documentation with the word plugin, and the one with the ID of 19 is the highest ranking result. Let’s see which document that is:

mysql> SELECT title FROM documents WHERE id = 19;
+----------------------------------------------------+
| title                                              |
+----------------------------------------------------+
| Dancer::Plugin - helper for writing Dancer plugins |
+----------------------------------------------------+

It’s the documentation for Dancer::Plugin, and it makes total sense that this is the first result for the word plugin. Sphinx setup is thus ready and we can get to the web application part of our little project.

(more…)

Supporting Perl Dancer and the “Last Chance” Foundation

Saturday, June 30th, 2012

Last month I set in motion the plan to make regular donations to open source projects and other good causes, and it’s time to do it again.

The choice of project that I’m donating to this month comes rather naturally, as it’s the project that gained most of my attention in the past twelve months, and which had a significant impact on my career — the Perl Dancer framework. Today is exactly one year since I have finished porting my website to Dancer, which was my very first experience with the framework, and through the months that followed I have used it in a number of real-life projects, developed a few plugins, and got a great contract job thanks to being experienced with it. Thank you Dancer for a great year!

The second donation that I’m making this month goes to “Last Chance”, a foundation that runs an animal shelter near Rawa Mazowiecka, my home town. I’m virtually adopting one of the dogs in the shelter — well, the choice was pretty straightforward when I saw that there was one dog that shared the first name with me! Stay strong, my dog namesake, maybe I’ll visit you one day when I’m in the neighborhood.

  • Archives

  • Categories

  • Meta

  • Latest Tweets


    Warning: Illegal string offset 'last_access' in /usr/local/www/odyniec.net/public/blog/wp-content/plugins/twitget/twitget.php on line 334

    Warning: Illegal string offset 'time_limit' in /usr/local/www/odyniec.net/public/blog/wp-content/plugins/twitget/twitget.php on line 334

    Warning: Illegal string offset 'last_access' in /usr/local/www/odyniec.net/public/blog/wp-content/plugins/twitget/twitget.php on line 336

    Warning: Illegal string offset 'twitter_api' in /usr/local/www/odyniec.net/public/blog/wp-content/plugins/twitget/twitget.php on line 234

    Warning: Illegal string offset 'user_token' in /usr/local/www/odyniec.net/public/blog/wp-content/plugins/twitget/twitget.php on line 262

    Warning: Illegal string offset 'user_secret' in /usr/local/www/odyniec.net/public/blog/wp-content/plugins/twitget/twitget.php on line 263

    Warning: Illegal string offset 'consumer_key' in /usr/local/www/odyniec.net/public/blog/wp-content/plugins/twitget/twitget.php on line 264

    Warning: Illegal string offset 'consumer_secret' in /usr/local/www/odyniec.net/public/blog/wp-content/plugins/twitget/twitget.php on line 265

    Warning: Illegal string offset 'twitter_username' in /usr/local/www/odyniec.net/public/blog/wp-content/plugins/twitget/twitget.php on line 270

    Warning: Illegal string offset 'show_retweets' in /usr/local/www/odyniec.net/public/blog/wp-content/plugins/twitget/twitget.php on line 272

    Warning: Illegal string offset 'exclude_replies' in /usr/local/www/odyniec.net/public/blog/wp-content/plugins/twitget/twitget.php on line 275

    Warning: Illegal string offset 'twitter_data' in /usr/local/www/odyniec.net/public/blog/wp-content/plugins/twitget/twitget.php on line 282

    Warning: Illegal string offset 'twitter_data' in /usr/local/www/odyniec.net/public/blog/wp-content/plugins/twitget/twitget.php on line 350

    Warning: Illegal string offset 'twitter_data' in /usr/local/www/odyniec.net/public/blog/wp-content/plugins/twitget/twitget.php on line 351
    Twitter outputted an error:
    .
    Warning: Illegal string offset 'time_format' in /usr/local/www/odyniec.net/public/blog/wp-content/plugins/twitget/twitget.php on line 484
  • Follow odyniec on Twitter