labs

http://linkfluence.net/
I'm a lumberjaph: Github Communities Posters for shipping

Github Poster

I've finally found a printer that can do international shipping for reasonable costs. The price will be 65€, and a paypal account will be setup soon.

So, if you're interested by a poster in A1 size, send me a mail. I'll need at least 15 persons to be able to do this. So contact me, and I will keep you informed.

By: franck on 2010-06-30T20:09:10

Tags: github

permalink

tidying the web with a teaspoon » forecasting: No www !

Grâce à Sunfox (avec lequel je ne suis pas complètement d’accord : il est un peu extrême dans la suppression pure et simple du TLD), je découvre, le site le plus utile au web : no-www.org !

Au passage, si vous avez raté la conversation sur les shorteners d’URL, ça a commencé chez Twitter qui ferait mieux de rajouter un attribut url à leur modèle NoSQL. A propos, il faudra qu’on m’explique quel peut-être le modèle d’une startup fabriquant un shortener, surtout quand elle adhère au 301works ; se faire racheter avant de dépasser les 20 caractères ?

I'm a lumberjaph: presque

presque

I've added a few new features to presque.

presque is a persistant job queue based on Redis and Tatsumaki.

A short list of current features implemented:

  • jobs are JSON object
  • possibility to stop/start queues
  • jobs can be delayed to run after a certain date in the future
  • workers can register themself, doing this, you can know when a worker started, what he have done, ...
  • statistics about queue, jobs, and workers
  • possible to store and fetch jobs in batch
  • a job can be unique

The REST interface is simple, and there is only a few methods. It's fast (I will provide numbers soon from our production environment), and workers can be implemented in any languages.

There have been a lot of refactoring lately. The main features missing right now are a simple HTML interface that will display various informations, pulling the data from the REST API (hint : if someone want to help to design this one ... :) ), websocket (sending a message to all workers).

There is a Perl client to the REST API: net::presque, that you can use with net::http::console:

perl bin/http-console --api_lib Net::Presque --url http://localhost:5000
http://localhost:5000> fetch_job {"queue_name":"twitter_stream"}
{
   "text" : "Australias new prime minister - julia gillard is our 27th prime minister.",
   "user" : "Lov3LifeAlways"
}

I've also wrote a better worker for Perl. It's a Moose::Role that you apply to your class. You need to write a work method, and your done. This worker handle retries, provide a logger, ... As for resque, there is two dispatcher:

  • normal : the worker grab a job, process it, then ask for the next job
  • fork : the worker grab a job, fork, let the child do the job and exit, while the parent ask for the next job. As resque says, "Resque assumes chaos". And me too, I like (ordered) chaos

I hope to finish the documentation and to writes one or two more workers as example (maybe in Python and javascript/node.js) soon to be able to tag a first version, and to collect some info about how many jobs have been processed at work (we use it to do url resolution and collect twitter data among few other things). Although I'm not sure I will release it to CPAN.

By: franck on 2010-06-25T08:59:53

Tags: presque redis

permalink

tidying the web with a teaspoon » forecasting: Enfin une factory avec Moose :)

Youpi ! MooseX::AbstractFactory

I'm a lumberjaph: Monthly Dancer meeting

A Monthly Dancer meeting

I've been contributing to Dancer for a few months now, and the discussions occurs mainly on IRC (irc.perl.org, #dancer) or on the Mailing List.

Last weekend, I had the occasion to meet sukria during the French Perl Workshop. This has been really productive, we had the occasion to talk about Plack, the templating system, websocket, ... and I really think we should have met before. It was also the occasion to meet another contributor, eiro, with whom I've been able to share some knowledge about Plack.

During the workshop, I made a talk about Plack and the Middlewares. The direct result of this is the last feature added by sukria: middlewares can be set in the configuration file of your Dancer application, and will be loaded for you.

The next release of Dancer won't generate an app.psgi file anymore, so you will only need to edit your environment file (like deployement.yaml), and add the following configuration:

warnings: 1
auto_reload: 1
plack_middlewares:
  Debug:
    - panels
    -
      - Response
      - Dancer::Version
      - Dancer::Settings

and your application will load some Plack::Middleware::Debug and Dancer::Debug panels.

Sukria has suggested a monthly drinkup meeting for people in/near Paris, to talk about Dancer and Plack, in a pub or another place where we can bring a laptop, have some beers and share idea/codes and other exchange technicals thoughts.

I hope to meet more Dancer developers and users in a near future (sawyer at Pise maybe ?).

By: franck cuny on 2010-06-20T16:18:23

Tags: Perl Dancer

permalink

I'm a lumberjaph: FPW 2010 summary

my small French Perl Workshop summary

First, no more welsh. Ever.

Even if Calais was not the easiest destination for every one, it was a really fun and intersting two days. I met nice folks, had great discussions and drink good beers.

For those who missed this workshop, a short summary for you:

  • had excellent discussions with sukria about the future of Dancer
  • fun facts about time zone and unicode with maddingue, rgs and fperrad
  • interesting chat with fperrad about lua and parrot
  • convinced more people that Plack is the future for Perl Web development
  • beers, and more beers with sukria, jerome, arnaud, rgs, camille and stephane.
  • talked with Marc about Plack, Dancer, and other stuff

    diner

My slides are available online (in french):

And to finish, two other summaries: sukria's one and twitter's timeline; and some pictures on flickr..

Thanks to laurent and sebastien for their hard work on organizing this conference.

By: franck on 2010-06-13T11:06:20

Tags: fpw perl conference

permalink

I'm a lumberjaph: Moosex::Net::API - update

MooseX::Net::API - update

MooseX::Net::API is a module to help writing clients for RESTful (and even non-RESTful) WebServices:

package my::api;
use MooseX::Net::API;

net_api_declare myapi => (
    api_base_url => 'http://api....',
    api_format   => 'json',
);

net_api_method users => (
    method      => 'GET',
    path        => '/users/:country',
    description => 'fetch a list of users',
    params      => [qw/country/],
    expected    => [qw/200 404/],
);

We've been using this module at work for the last few months on various internal APIs, and I'm pretty pleased with the result so far.

Lately I've started to rework the core. I've tried to split most of the functionalities into roles, and rework the code that generates the various methods. I've also added methods to access miscellaneous information :

my $client = my::api->new;

# to get a list of API methods
$client->meta->get_all_net_api_methods();

# find an API method
my $method = $client->meta->find_net_api_method_by_name('users');

# and now informations about the method
say $method->documentation;

name:        users
description: fetch a list of useres
method:      GET
path:        /users/:country
arguments:   country
expected:    200, 404

It's not yet complete, but a new version will be available soon on CPAN. Here is a list of some more features I plan to add quickly:

  • better internal API
  • better authorization support (OAuth!)
  • add more methods to provide better introspection
  • better unserialization
  • more tests and better documentation
  • generate POD via a PODWeaver plugin ?
  • plugins ?
  • renaming ? (not sure it really fits in the MooseX:: namespace)

http-console

I've also started Net::HTTP::Console. It's inspired by http-console. It relies on MX::Net::API, and can use any libraries written with MX::Net::API, as well as any raw RESTful API. As an example, let's use it on twitter.

http-console --url http://api.twitter.com --format json

http://127.0.0.1:5984> GET /1/statuses/public_timeline
[
    {
      "source" : "web",
      "favorited" : false,
      "geo" : null,
      "coordinates" : null,
      "place" : null,
      ...
    }
]

http://127.0.0.1:5984> show headers
cache-control: no-cache, no-store, must-revalidate, pre-check=0, post-check=0
last-modified: Mon, 07 Jun 2010 15:27:12 GMT
x-transaction: 1275924432-94882-31146
x-ratelimit-reset: 1275925258
...

You can call any method from the twitter API (at the exception of the ones that require authentication: it's not supported yet).

You can also use it with any library that uses MX::Net::API:

http-console --lib Net::Backtweet

http://api.backtweet.com> help command
available commands:
- tweets_by_url
- stats_by_url
- good_tweets_by_url

http://api.backtype.com> help command tweets_by_url
name:        tweets_by_url
description: Retrieve tweets that link to a given URL, whether the links are shortened or unshortened.
method:      GET
path:        /tweets/search/links

http://api.backtype.com> stats_by_url {"q":"http://lumberjaph.net","key":s3kr3t"}
{
   "tweetcount" : 388
}

Arguments to the methods are serialized in JSON format. Not sure if it's the best idea I will see if it needs improvement while using it. You can also perform POST and PUT with content.

http://localhost:5984> POST /test_rtgi_fetcher {"foo":"bar"}
{
   "ok" : true,
   "rev" : "1-fe67006eb0e02e5f0057b5b2a6672391",
   "id" : "fe3175615a34eb28153479307c000f26"
}

It's far from being complete at the moment, but I will extend it quickly. Right now, you can define global headers, and get help for all methods in your MX::Net::API library. Authentication is on top of my priority list, as is alias creation, so instead of doing (on a non-moosex::net::api lib):

GET /users/

you will do:

alias users/:country as users

then:

users {"country":"france"}

(and yes, I've switched from wordpress to blawd)

By: franck on 2010-06-10T11:55:00

Tags: moosex moose rest http-console

permalink

Erralt » LF: Make SFTP and umask working

In your file /etc/ssh/sshd_config use this line for SFTP working with custom umask :

Subsystem sftp /bin/sh -c 'umask 0002; exec /usr/lib/openssh/sftp-server'